GitlabCI build failed, docker.sock: write: broken pipe

I was trying to implement a new CI workflow using GitlabCI but now I can't understand the issue. I followed this doc and I choose the Docker socket binding method.

Everything is running on the same machine, a RancherOS :

$ uname -a    
Linux xxx 4.2.8-ckt4-rancher #1 SMP Tue Apr 5 16:12:00 UTC 2016 x86_64 x86_64 x86_64 GNU/Linux

$ docker ps
CONTAINER ID        IMAGE                         COMMAND                  CREATED             STATUS              PORTS                                      NAMES
57801c725226        gitlab/gitlab-runner:latest   "/usr/bin/dumb-init /"   14 hours ago        Up 14 hours                                                    gitlab-runner
50e3a9dabe77        gitlab/gitlab-ce              "/assets/wrapper"        17 hours ago        Up 17 hours         80/tcp, 443/tcp, 0.0.0.0:1022->22/tcp      gitlab
ed2b6d9a8ea7        jwilder/nginx-proxy           "/app/docker-entrypoi"   6 days ago          Up 3 days           0.0.0.0:80->80/tcp, 0.0.0.0:443->443/tcp   proxy

Run command of the runner's container

$ docker run -d --name gitlab-runner --restart always \
-v ~/docker/gitlab-runner/config:/etc/gitlab-runner \
-v /var/run/docker.sock:/var/run/docker.sock \
gitlab/gitlab-runner:latest

Runner's config

$ sudo cat ~/docker/gitlab-runner/config/config.toml 
concurrent = 1
check_interval = 0

[[runners]]
  name = "ITrancher, docker based runner 4"
  url = "https://git.xxx.com"
  token = "xxx"
  executor = "docker"
  [runners.docker]
    tls_verify = false
    image = "docker:latest"
    privileged = false
    disable_cache = false
    volumes = ["/var/run/docker.sock:/var/run/docker.sock", "/cache"]
  [runners.cache]
  [runners.kubernetes]
    host = ""
    cert_file = ""
    key_file = ""
    ca_file = ""
    image = ""
    namespace = ""
    privileged = false
    cpus = ""
    memory = ""
    service_cpus = ""
    service_memory = ""

Here is my .gitlab-ci.yml file

image: docker:latest
before_script:
- docker login -u gitlab-ci-token -p $CI_BUILD_TOKEN registry.xxx.com
stages:
- build
build:
  stage: build
  script:
  - docker build --pull -f Dockerfile-php -t registry.xxx.com/dev/kayako-php:latest .
  - docker push registry.xxx.com/dev/kayako-php:latest
  - docker build --pull -f Dockerfile-nginx -t registry.xxx.com/dev/kayako-nginx:latest .
  - docker push registry.xxx.com/dev/kayako-nginx:latest

And this is what I get into GitlabCI when the build is hooked

Running with gitlab-ci-multi-runner 1.6.1 (c52ad4f)
Using Docker executor with image docker:latest ...
ERROR: Failed to create container volume for /builds/dev write unix @->/var/run/docker.sock: write: broken pipe
ERROR: Preparation failed: write unix @->/var/run/docker.sock: write: broken pipe
Will be retried in 3s ...
Using Docker executor with image docker:latest ...
ERROR: Failed to create container volume for /builds/dev write unix @->/var/run/docker.sock: write: broken pipe
ERROR: Preparation failed: write unix @->/var/run/docker.sock: write: broken pipe
Will be retried in 3s ...
Using Docker executor with image docker:latest ...
ERROR: Failed to create container volume for /builds/dev write unix @->/var/run/docker.sock: write: broken pipe
ERROR: Preparation failed: write unix @->/var/run/docker.sock: write: broken pipe
Will be retried in 3s ...
ERROR: Build failed (system failure): write unix @->/var/run/docker.sock: write: broken pipe

And this is the output when I try to run the build locally with --debug flag inside my gitlab-runner container.

# gitlab-runner --debug exec docker build
Runtime platform                                    arch=amd64 os=linux revision=c52ad4f version=1.6.1
Running with gitlab-ci-multi-runner 1.6.1 (c52ad4f)
Shell configuration: environment: []
dockercommand:
- sh
- -c
- "if [ -x /usr/local/bin/bash ]; then\n\texec /usr/local/bin/bash \nelif [ -x /usr/bin/bash
  ]; then\n\texec /usr/bin/bash \nelif [ -x /bin/bash ]; then\n\texec /bin/bash \nelif
  [ -x /usr/local/bin/sh ]; then\n\texec /usr/local/bin/sh \nelif [ -x /usr/bin/sh
  ]; then\n\texec /usr/bin/sh \nelif [ -x /bin/sh ]; then\n\texec /bin/sh \nelse\n\techo
  shell not found\n\texit 1\nfi\n\n"
command: bash
arguments: []
passfile: false
extension: ""
  build=1 project=1
Using Docker executor with image docker:latest ...
Applying docker.Client transport fix: &{false 0xc820306810 <nil> 0xc820238320 unix:///var/run/docker.sock 0xc820232780 0xc820340ba0 [1 18] [] [] <nil>}  host=unix:///var/run/docker.sock
Creating build volume...                            build=1 project=1
Looking for prebuilt image gitlab-runner-prebuilt-x86_64:c52ad4f ...  build=1 project=1
Loading prebuilt image...                           build=1 project=1
Closed all idle connections for docker.Client: &{false 0xc820306900 <nil> 0x39d9ee0 unix:///var/run/docker.sock 0xc820232780 0xc820340ba0 [1 18] [1 22] [1 18] 0xc820306930} 
ERROR: Preparation failed: write unix @->/var/run/docker.sock: write: broken pipe
Will be retried in 3s ...
Shell configuration: environment: []
dockercommand:
- sh
- -c
- "if [ -x /usr/local/bin/bash ]; then\n\texec /usr/local/bin/bash \nelif [ -x /usr/bin/bash
  ]; then\n\texec /usr/bin/bash \nelif [ -x /bin/bash ]; then\n\texec /bin/bash \nelif
  [ -x /usr/local/bin/sh ]; then\n\texec /usr/local/bin/sh \nelif [ -x /usr/bin/sh
  ]; then\n\texec /usr/bin/sh \nelif [ -x /bin/sh ]; then\n\texec /bin/sh \nelse\n\techo
  shell not found\n\texit 1\nfi\n\n"
command: bash
arguments: []
passfile: false
extension: ""
  build=1 project=1
Using Docker executor with image docker:latest ...
Applying docker.Client transport fix: &{false 0xc820306900 <nil> 0x39d9ee0 unix:///var/run/docker.sock 0xc820232780 0xc820340ba0 [1 18] [1 22] [1 18] 0xc820306930}  host=unix:///var/run/docker.sock
Creating build volume...                            build=1 project=1
Looking for prebuilt image gitlab-runner-prebuilt-x86_64:c52ad4f ...  build=1 project=1
Loading prebuilt image...                           build=1 project=1
Closed all idle connections for docker.Client: &{false 0xc8203072f0 <nil> 0x39d9ee0 unix:///var/run/docker.sock 0xc820232780 0xc820340ba0 [1 18] [1 22] [1 18] 0xc820306930} 
ERROR: Preparation failed: write unix @->/var/run/docker.sock: write: broken pipe
Will be retried in 3s ...
Shell configuration: environment: []
dockercommand:
- sh
- -c
- "if [ -x /usr/local/bin/bash ]; then\n\texec /usr/local/bin/bash \nelif [ -x /usr/bin/bash
  ]; then\n\texec /usr/bin/bash \nelif [ -x /bin/bash ]; then\n\texec /bin/bash \nelif
  [ -x /usr/local/bin/sh ]; then\n\texec /usr/local/bin/sh \nelif [ -x /usr/bin/sh
  ]; then\n\texec /usr/bin/sh \nelif [ -x /bin/sh ]; then\n\texec /bin/sh \nelse\n\techo
  shell not found\n\texit 1\nfi\n\n"
command: bash
arguments: []
passfile: false
extension: ""
  build=1 project=1
Using Docker executor with image docker:latest ...
Applying docker.Client transport fix: &{false 0xc8203072f0 <nil> 0x39d9ee0 unix:///var/run/docker.sock 0xc820232780 0xc820340ba0 [1 18] [1 22] [1 18] 0xc820306930}  host=unix:///var/run/docker.sock
Creating build volume...                            build=1 project=1
Looking for prebuilt image gitlab-runner-prebuilt-x86_64:c52ad4f ...  build=1 project=1
Loading prebuilt image...                           build=1 project=1
Closed all idle connections for docker.Client: &{false 0xc820307a10 <nil> 0x39d9ee0 unix:///var/run/docker.sock 0xc820232780 0xc820340ba0 [1 18] [1 22] [1 18] 0xc820306930} 
ERROR: Preparation failed: write unix @->/var/run/docker.sock: write: broken pipe
Will be retried in 3s ...
ERROR: Build failed (system failure): write unix @->/var/run/docker.sock: write: broken pipe
FATAL: write unix @->/var/run/docker.sock: write: broken pipe

But my docker.sock binding seems to be ok :

# echo -e "GET /containers/json HTTP/1.0\r\n" | nc -U /var/run/docker.sock
HTTP/1.0 200 OK
Content-Type: application/json
Server: Docker/1.10.3 (linux)
Date: Wed, 19 Oct 2016 10:22:58 GMT

[{json is ok}]

Anyone has an idea ?

It's a known bug with latest versions of gitlab-ci-multi-runner (starting at v1.5.0).

You can use version 1.4.2 until the bug is properly fixed. Here is your docker run command using the latest working version:

$ docker run -d --name gitlab-runner --restart always \
-v ~/docker/gitlab-runner/config:/etc/gitlab-runner \
-v /var/run/docker.sock:/var/run/docker.sock \
gitlab/gitlab-runner:v1.4.2

An issue already exists on Gitlab CI Multi Runner repository: https://gitlab.com/gitlab-org/gitlab-ci-multi-runner/issues/1617

Edit: Please use the v1.4.2 instead of v1.4.3. The v1.4.3 is wrongly tagged on Docker Hub:

$ docker run --rm -it gitlab/gitlab-runner:v1.4.3 -v
Version:      1.5.2
Git revision: 76fdacd
Git branch:   1-5-stable
GO version:   go1.6.3
Built:        Wed, 24 Aug 2016 13:35:10 +0000
OS/Arch:      linux/amd64

this question is pretty old and while the original question might have been answered and working with the suggested workaround is it currently NOT WORKING with version 11.0.0 of the gitlab-runner. I was able to get to work...

OS: rancheros 1.4.0 console: alpine Docker: docker-17.12.1-ce && docker-18.03.1-ce

I installed the runner as usual, then registered. Discovered the failure... then edited the toml file and changed privileged = false to privileged = true.

sudo vim /srv/gitlab-runner/config/config.toml docker stop gitlab-runner docker rm gitlab-runner docker run -d --privileged --name gitlab-runner --restart always -v /srv/gitlab-runner/config:/etc/gitlab-runner -v /var/run/docker.sock:/var/run/docker.sock gitlab/gitlab-runner:latest

来源：https://stackoverflow.com/questions/40126748/gitlabci-build-failed-docker-sock-write-broken-pipe

标签

Docker

gitlab-ci

gitlab-ci-runner

Rancher