问题
Short story: I have a gitlab-runner in a docker-compose and I want to be able to use DIND, but I'm facing some difficulties....
I try to create a platform which contains :
- a sonar
- gitlab ce
- gitlab runner
- a registry
Theses services are started and managed by docker-compose I use the gitlab-ci to verify tests, coverage and create a docker image which is uploaded to the registry I have a single shared runner which work for testing purpose
here is the config.toml :
concurrent = 1
check_interval = 0
[session_server]
session_timeout = 1800
[[runners]]
name = "21fbd75383fe"
url = "http://gitlab/ci"
token = "--"
executor = "docker"
[runners.custom_build_dir]
[runners.docker]
tls_verify = false
image = "docker:latest"
privileged = true
disable_entrypoint_overwrite = false
oom_kill_disable = false
disable_cache = false
volumes = ["/cache"]
shm_size = 0
network_mode = "oral_default"
[runners.cache]
[runners.cache.s3]
[runners.cache.gcs]
[runners.custom]
run_exec = ""
My problem : I need to specify in the runner : network_mode = "oral_default" in order to clone the reposotory. But if I want to use Dind to build my image, I get this error :
ealth check container logs:
2019-08-20T14:11:21.847061412Z FATAL: No HOST or PORT found
Service container logs:
2019-08-20T14:11:18.189776447Z Generating RSA private key, 4196 bit long modulus (2 primes)
2019-08-20T14:11:18.495587062Z .......................................++++
2019-08-20T14:11:19.261799191Z ...............................................................................................++++
2019-08-20T14:11:19.262353078Z e is 65537 (0x010001)
2019-08-20T14:11:19.288253880Z Generating RSA private key, 4196 bit long modulus (2 primes)
2019-08-20T14:11:19.735803254Z .......................................................++++
2019-08-20T14:11:20.998049980Z .........................................................................................................................................................++++
2019-08-20T14:11:20.998511667Z e is 65537 (0x010001)
2019-08-20T14:11:21.040579379Z Signature ok
2019-08-20T14:11:21.040598512Z subject=CN = docker:dind server
2019-08-20T14:11:21.040814852Z Getting CA Private Key
2019-08-20T14:11:21.071374613Z /certs/server/cert.pem: OK
2019-08-20T14:11:21.075263091Z Generating RSA private key, 4196 bit long modulus (2 primes)
2019-08-20T14:11:21.159644328Z .........++++
2019-08-20T14:11:22.011823318Z ..............................................................................................................++++
2019-08-20T14:11:22.012330364Z e is 65537 (0x010001)
2019-08-20T14:11:22.046700923Z Signature ok
2019-08-20T14:11:22.046735711Z subject=CN = docker:dind client
2019-08-20T14:11:22.046961229Z Getting CA Private Key
2019-08-20T14:11:22.067938238Z /certs/client/cert.pem: OK
2019-08-20T14:11:22.099482505Z time="2019-08-20T14:11:22.099370855Z" level=info msg="Starting up"
2019-08-20T14:11:22.100758237Z time="2019-08-20T14:11:22.100680440Z" level=warning msg="could not change group /var/run/docker.sock to docker: group docker not found"
2019-08-20T14:11:22.101707958Z time="2019-08-20T14:11:22.101626009Z" level=info msg="libcontainerd: started new containerd process" pid=54
2019-08-20T14:11:22.101727175Z time="2019-08-20T14:11:22.101657983Z" level=info msg="parsed scheme: \"unix\"" module=grpc
2019-08-20T14:11:22.101733998Z time="2019-08-20T14:11:22.101673740Z" level=info msg="scheme \"unix\" not registered, fallback to default scheme" module=grpc
2019-08-20T14:11:22.101750834Z time="2019-08-20T14:11:22.101693854Z" level=info msg="ccResolverWrapper: sending update to cc: {[{unix:///var/run/docker/containerd/containerd.sock 0 <nil>}] }" module=grpc
2019-08-20T14:11:22.101758034Z time="2019-08-20T14:11:22.101710395Z" level=info msg="ClientConn switching balancer to \"pick_first\"" module=grpc
2019-08-20T14:11:22.101883362Z time="2019-08-20T14:11:22.101777690Z" level=info msg="pickfirstBalancer: HandleSubConnStateChange: 0xc0008181f0, CONNECTING" module=grpc
2019-08-20T14:11:22.119465945Z time="2019-08-20T14:11:22.119356782Z" level=info msg="starting containerd" revision=894b81a4b802e4eb2a91d1ce216b8817763c29fb version=v1.2.6
2019-08-20T14:11:22.119997814Z time="2019-08-20T14:11:22.119921726Z" level=info msg="loading plugin "io.containerd.content.v1.content"..." type=io.containerd.content.v1
2019-08-20T14:11:22.120066267Z time="2019-08-20T14:11:22.120010967Z" level=info msg="loading plugin "io.containerd.snapshotter.v1.btrfs"..." type=io.containerd.snapshotter.v1
2019-08-20T14:11:22.120297760Z time="2019-08-20T14:11:22.120239139Z" level=warning msg="failed to load plugin io.containerd.snapshotter.v1.btrfs" error="path /var/lib/docker/containerd/daemon/io.containerd.snapshotter.v1.btrfs must be a btrfs filesystem to be used with the btrfs snapshotter"
2019-08-20T14:11:22.120305857Z time="2019-08-20T14:11:22.120253119Z" level=info msg="loading plugin "io.containerd.snapshotter.v1.aufs"..." type=io.containerd.snapshotter.v1
2019-08-20T14:11:22.124698054Z time="2019-08-20T14:11:22.124622589Z" level=warning msg="failed to load plugin io.containerd.snapshotter.v1.aufs" error="modprobe aufs failed: "ip: can't find device 'aufs'\naufs 241664 0 \nmodprobe: can't change directory to '/lib/modules': No such file or directory\n": exit status 1"
2019-08-20T14:11:22.124716529Z time="2019-08-20T14:11:22.124642302Z" level=info msg="loading plugin "io.containerd.snapshotter.v1.native"..." type=io.containerd.snapshotter.v1
2019-08-20T14:11:22.124759418Z time="2019-08-20T14:11:22.124715546Z" level=info msg="loading plugin "io.containerd.snapshotter.v1.overlayfs"..." type=io.containerd.snapshotter.v1
2019-08-20T14:11:22.124901964Z time="2019-08-20T14:11:22.124862487Z" level=info msg="loading plugin "io.containerd.snapshotter.v1.zfs"..." type=io.containerd.snapshotter.v1
2019-08-20T14:11:22.125128168Z time="2019-08-20T14:11:22.125083244Z" level=warning msg="failed to load plugin io.containerd.snapshotter.v1.zfs" error="path /var/lib/docker/containerd/daemon/io.containerd.snapshotter.v1.zfs must be a zfs filesystem to be used with the zfs snapshotter"
2019-08-20T14:11:22.125137429Z time="2019-08-20T14:11:22.125095730Z" level=info msg="loading plugin "io.containerd.metadata.v1.bolt"..." type=io.containerd.metadata.v1
2019-08-20T14:11:22.125191366Z time="2019-08-20T14:11:22.125143058Z" level=warning msg="could not use snapshotter aufs in metadata plugin" error="modprobe aufs failed: "ip: can't find device 'aufs'\naufs 241664 0 \nmodprobe: can't change directory to '/lib/modules': No such file or directory\n": exit status 1"
2019-08-20T14:11:22.125200443Z time="2019-08-20T14:11:22.125154226Z" level=warning msg="could not use snapshotter zfs in metadata plugin" error="path /var/lib/docker/containerd/daemon/io.containerd.snapshotter.v1.zfs must be a zfs filesystem to be used with the zfs snapshotter"
2019-08-20T14:11:22.125205718Z time="2019-08-20T14:11:22.125160660Z" level=warning msg="could not use snapshotter btrfs in metadata plugin" error="path /var/lib/docker/containerd/daemon/io.containerd.snapshotter.v1.btrfs must be a btrfs filesystem to be used with the btrfs snapshotter"
2019-08-20T14:11:22.299853510Z time="2019-08-20T14:11:22.299730279Z" level=info msg="loading plugin "io.containerd.differ.v1.walking"..." type=io.containerd.differ.v1
2019-08-20T14:11:22.299878846Z time="2019-08-20T14:11:22.299776167Z" level=info msg="loading plugin "io.containerd.gc.v1.scheduler"..." type=io.containerd.gc.v1
2019-08-20T14:11:22.299887790Z time="2019-08-20T14:11:22.299812949Z" level=info msg="loading plugin "io.containerd.service.v1.containers-service"..." type=io.containerd.service.v1
2019-08-20T14:11:22.299904150Z time="2019-08-20T14:11:22.299828135Z" level=info msg="loading plugin "io.containerd.service.v1.content-service"..." type=io.containerd.service.v1
2019-08-20T14:11:22.299913745Z time="2019-08-20T14:11:22.299842184Z" level=info msg="loading plugin "io.containerd.service.v1.diff-service"..." type=io.containerd.service.v1
2019-08-20T14:11:22.299921184Z time="2019-08-20T14:11:22.299854806Z" level=info msg="loading plugin "io.containerd.service.v1.images-service"..." type=io.containerd.service.v1
2019-08-20T14:11:22.299927659Z time="2019-08-20T14:11:22.299869296Z" level=info msg="loading plugin "io.containerd.service.v1.leases-service"..." type=io.containerd.service.v1
2019-08-20T14:11:22.299933467Z time="2019-08-20T14:11:22.299884994Z" level=info msg="loading plugin "io.containerd.service.v1.namespaces-service"..." type=io.containerd.service.v1
2019-08-20T14:11:22.299938406Z time="2019-08-20T14:11:22.299904463Z" level=info msg="loading plugin "io.containerd.service.v1.snapshots-service"..." type=io.containerd.service.v1
2019-08-20T14:11:22.299943250Z time="2019-08-20T14:11:22.299917532Z" level=info msg="loading plugin "io.containerd.runtime.v1.linux"..." type=io.containerd.runtime.v1
2019-08-20T14:11:22.300179457Z time="2019-08-20T14:11:22.300128875Z" level=info msg="loading plugin "io.containerd.runtime.v2.task"..." type=io.containerd.runtime.v2
2019-08-20T14:11:22.300316944Z time="2019-08-20T14:11:22.300270682Z" level=info msg="loading plugin "io.containerd.monitor.v1.cgroups"..." type=io.containerd.monitor.v1
2019-08-20T14:11:22.300745465Z time="2019-08-20T14:11:22.300693221Z" level=info msg="loading plugin "io.containerd.service.v1.tasks-service"..." type=io.containerd.service.v1
2019-08-20T14:11:22.300776133Z time="2019-08-20T14:11:22.300731401Z" level=info msg="loading plugin "io.containerd.internal.v1.restart"..." type=io.containerd.internal.v1
2019-08-20T14:11:22.300819617Z time="2019-08-20T14:11:22.300782007Z" level=info msg="loading plugin "io.containerd.grpc.v1.containers"..." type=io.containerd.grpc.v1
2019-08-20T14:11:22.300828421Z time="2019-08-20T14:11:22.300797250Z" level=info msg="loading plugin "io.containerd.grpc.v1.content"..." type=io.containerd.grpc.v1
2019-08-20T14:11:22.300840355Z time="2019-08-20T14:11:22.300809287Z" level=info msg="loading plugin "io.containerd.grpc.v1.diff"..." type=io.containerd.grpc.v1
2019-08-20T14:11:22.300845835Z time="2019-08-20T14:11:22.300821506Z" level=info msg="loading plugin "io.containerd.grpc.v1.events"..." type=io.containerd.grpc.v1
2019-08-20T14:11:22.300863231Z time="2019-08-20T14:11:22.300835107Z" level=info msg="loading plugin "io.containerd.grpc.v1.healthcheck"..." type=io.containerd.grpc.v1
2019-08-20T14:11:22.300870180Z time="2019-08-20T14:11:22.300846235Z" level=info msg="loading plugin "io.containerd.grpc.v1.images"..." type=io.containerd.grpc.v1
2019-08-20T14:11:22.300899955Z time="2019-08-20T14:11:22.300858124Z" level=info msg="loading plugin "io.containerd.grpc.v1.leases"..." type=io.containerd.grpc.v1
2019-08-20T14:11:22.300910656Z time="2019-08-20T14:11:22.300868856Z" level=info msg="loading plugin "io.containerd.grpc.v1.namespaces"..." type=io.containerd.grpc.v1
2019-08-20T14:11:22.300924355Z time="2019-08-20T14:11:22.300885954Z" level=info msg="loading plugin "io.containerd.internal.v1.opt"..." type=io.containerd.internal.v1
2019-08-20T14:11:22.301165214Z time="2019-08-20T14:11:22.301127593Z" level=info msg="loading plugin "io.containerd.grpc.v1.snapshots"..." type=io.containerd.grpc.v1
2019-08-20T14:11:22.301173167Z time="2019-08-20T14:11:22.301148082Z" level=info msg="loading plugin "io.containerd.grpc.v1.tasks"..." type=io.containerd.grpc.v1
2019-08-20T14:11:22.301197447Z time="2019-08-20T14:11:22.301160478Z" level=info msg="loading plugin "io.containerd.grpc.v1.version"..." type=io.containerd.grpc.v1
2019-08-20T14:11:22.301208675Z time="2019-08-20T14:11:22.301172158Z" level=info msg="loading plugin "io.containerd.grpc.v1.introspection"..." type=io.containerd.grpc.v1
2019-08-20T14:11:22.301420074Z time="2019-08-20T14:11:22.301383826Z" level=info msg=serving... address="/var/run/docker/containerd/containerd-debug.sock"
2019-08-20T14:11:22.301510586Z time="2019-08-20T14:11:22.301457137Z" level=info msg=serving... address="/var/run/docker/containerd/containerd.sock"
2019-08-20T14:11:22.301521798Z time="2019-08-20T14:11:22.301472502Z" level=info msg="containerd successfully booted in 0.182717s"
2019-08-20T14:11:22.306618029Z time="2019-08-20T14:11:22.306496623Z" level=info msg="pickfirstBalancer: HandleSubConnStateChange: 0xc0008181f0, READY" module=grpc
2019-08-20T14:11:22.308604516Z time="2019-08-20T14:11:22.308507649Z" level=info msg="parsed scheme: \"unix\"" module=grpc
2019-08-20T14:11:22.308624244Z time="2019-08-20T14:11:22.308531988Z" level=info msg="scheme \"unix\" not registered, fallback to default scheme" module=grpc
2019-08-20T14:11:22.308630203Z time="2019-08-20T14:11:22.308550514Z" level=info msg="ccResolverWrapper: sending update to cc: {[{unix:///var/run/docker/containerd/containerd.sock 0 <nil>}] }" module=grpc
2019-08-20T14:11:22.308635654Z time="2019-08-20T14:11:22.308567856Z" level=info msg="ClientConn switching balancer to \"pick_first\"" module=grpc
2019-08-20T14:11:22.308694129Z time="2019-08-20T14:11:22.308627145Z" level=info msg="pickfirstBalancer: HandleSubConnStateChange: 0xc000707cd0, CONNECTING" module=grpc
2019-08-20T14:11:22.308731380Z time="2019-08-20T14:11:22.308648131Z" level=info msg="blockingPicker: the picked transport is not ready, loop back to repick" module=grpc
2019-08-20T14:11:22.308943521Z time="2019-08-20T14:11:22.308874942Z" level=info msg="pickfirstBalancer: HandleSubConnStateChange: 0xc000707cd0, READY" module=grpc
2019-08-20T14:11:22.309450117Z time="2019-08-20T14:11:22.309385625Z" level=info msg="parsed scheme: \"unix\"" module=grpc
2019-08-20T14:11:22.309462252Z time="2019-08-20T14:11:22.309404366Z" level=info msg="scheme \"unix\" not registered, fallback to default scheme" module=grpc
2019-08-20T14:11:22.309467958Z time="2019-08-20T14:11:22.309419574Z" level=info msg="ccResolverWrapper: sending update to cc: {[{unix:///var/run/docker/containerd/containerd.sock 0 <nil>}] }" module=grpc
2019-08-20T14:11:22.309473276Z time="2019-08-20T14:11:22.309431644Z" level=info msg="ClientConn switching balancer to \"pick_first\"" module=grpc
2019-08-20T14:11:22.309568429Z time="2019-08-20T14:11:22.309500963Z" level=info msg="pickfirstBalancer: HandleSubConnStateChange: 0xc000144fa0, CONNECTING" module=grpc
2019-08-20T14:11:22.309585745Z time="2019-08-20T14:11:22.309506179Z" level=info msg="blockingPicker: the picked transport is not ready, loop back to repick" module=grpc
2019-08-20T14:11:22.309786808Z time="2019-08-20T14:11:22.309719559Z" level=info msg="pickfirstBalancer: HandleSubConnStateChange: 0xc000144fa0, READY" module=grpc
2019-08-20T14:11:22.749050188Z time="2019-08-20T14:11:22.748856365Z" level=warning msg="Your kernel does not support swap memory limit"
2019-08-20T14:11:22.749090607Z time="2019-08-20T14:11:22.748905994Z" level=warning msg="Your kernel does not support cgroup rt period"
2019-08-20T14:11:22.749100435Z time="2019-08-20T14:11:22.748934597Z" level=warning msg="Your kernel does not support cgroup rt runtime"
2019-08-20T14:11:22.749424856Z time="2019-08-20T14:11:22.749289206Z" level=info msg="Loading containers: start."
2019-08-20T14:11:22.760083557Z time="2019-08-20T14:11:22.759917977Z" level=warning msg="Running modprobe bridge br_netfilter failed with message: ip: can't find device 'bridge'\nbridge 151552 1 br_netfilter\nstp 16384 1 bridge\nllc 16384 2 bridge,stp\nip: can't find device 'br_netfilter'\nbr_netfilter 24576 0 \nbridge 151552 1 br_netfilter\nmodprobe: can't change directory to '/lib/modules': No such file or directory\n, error: exit status 1"
2019-08-20T14:11:22.766459849Z time="2019-08-20T14:11:22.766314726Z" level=warning msg="Running modprobe nf_nat failed with message: `ip: can't find device 'nf_nat'\nnf_nat_masquerade_ipv4 16384 1 ipt_MASQUERADE\nnf_nat_ipv4 16384 1 iptable_nat\nnf_nat 32768 3 xt_nat,nf_nat_masquerade_ipv4,nf_nat_ipv4\nnf_conntrack 131072 9 ip_vs,xt_nat,ipt_MASQUERADE,nf_nat_masquerade_ipv4,nf_conntrack_netlink,nf_conntrack_ipv4,nf_nat_ipv4,xt_conntrack,nf_nat\nlibcrc32c 16384 3 ip_vs,nf_nat,nf_conntrack\nmodprobe: can't change directory to '/lib/modules': No such file or directory`, error: exit status 1"
2019-08-20T14:11:22.772066324Z time="2019-08-20T14:11:22.771952709Z" level=warning msg="Running modprobe xt_conntrack failed with message: `ip: can't find device 'xt_conntrack'\nxt_conntrack 16384 8 \nnf_conntrack 131072 9 ip_vs,xt_nat,ipt_MASQUERADE,nf_nat_masquerade_ipv4,nf_conntrack_netlink,nf_conntrack_ipv4,nf_nat_ipv4,xt_conntrack,nf_nat\nx_tables 40960 11 xt_statistic,ipt_REJECT,xt_comment,xt_mark,xt_nat,xt_tcpudp,ipt_MASQUERADE,xt_addrtype,iptable_filter,xt_conntrack,ip_tables\nmodprobe: can't change directory to '/lib/modules': No such file or directory`, error: exit status 1"
*********
Pulling docker image docker:latest ...
Using docker image sha256:9a38a85b1e4e7bb53b7c7cc45afff6ba7b1cdfe01b9738f36a3b4ad0cdb10b00 for docker:latest ...
Running on runner-sbVCrx6S-project-1-concurrent-0 via 0937d4b8d68a...
Fetching changes with git depth set to 50...
Reinitialized existing Git repository in /builds/root/oral/.git/
From http://gitlab/root/oral
b1b2ac2..c0bb7ea master -> origin/master
Checking out c0bb7ea6 as master...
Skipping Git submodules setup
$ docker info
Client:
Debug Mode: false
Server:
ERROR: error during connect: Get http://docker:2375/v1.40/info: dial tcp: lookup docker on 127.0.0.11:53: no such host
Here is my gitlab ci:
image: python:3.6-stretch
stages:
- test
- sonar
- upload to registry
.test:
stage: test
cache:
paths:
- ~/.cache/
artifacts:
untracked: true
script:
- pip install -r requirement.txt
- python -m pytest
- python -m pytest --cov=src --cov-report=xml
.sonar:
image: zaquestion/sonarqube-scanner
dependencies:
- test
stage: sonar
script:
- sonar-scanner
upload to registry:
image: docker:latest
stage: upload to registry
services:
- docker:dind
script:
- docker info
- docker build -t local_image_oral:latest
- docker push local_image_oral:latest
- docker tag local_image_oral:latest registry:5000/local_image_oral:latest
- docker push registry:5000/local_image_oral
I strongly think, that the spwaned container is in another network and so can't access the docker-compose network which lead to this behaviour.
Can you help me ? Thank You.
回答1:
@MitsiDev this can happen while using newer docker image like version 19.03. Although this issue is a bit old, but the problem persists till today.
I faced the issue recently and turns out there is a known "solution" or workaround however.
Refer to this Release Note, if you want more details.
Reason:
As of version 19.03, docker:dind will automatically generate TLS certificates and require using them for communication. This is from Docker's official documentation:
Solution/workarounds:
According to the Release Note, there are 2 workarounds:
- Explicitly turn off TLS.
- Configure GitLab Runner to use TLS.
Turn off TLS
If you cannot or do not want to edit the config.toml, like when running jobs in a gitlab shared runner:
.gitlab-ci.yml:
image: docker:19.03
variables:
DOCKER_TLS_CERTDIR: ""
Configure TLS
If you are running jobs on a runner that you have write access to the config.toml file (and you know what you are doing):
config.toml:
...
[[runners]]
name = "My Docker Runner"
url = "http://gitlab.com"
token = ""
executor = "docker"
[runners.custom_build_dir]
[runners.docker]
## Changes here ##
privileged = true ## may not be necessary, default is false
volumes = ["/certs/client", "/cache"] ## adds volume "/certs/client"
shm_size = 0
...
.gitlab-ci.yml:
image: docker:19.03
variables:
# Create the certificates inside this directory for both the server
# and client. The certificates used by the client will be created in
# /certs/client so we only need to share this directory with the
# volume mount in `config.toml`.
DOCKER_TLS_CERTDIR: "/certs"`
来源：https://stackoverflow.com/questions/57576083/gitlab-runner-in-docker-compose-cannot-connect-http-docker2375-v1-40-info