How do I access a private Docker registry with a self signed certificate using Kubernetes?

Question

Currently, running a private Docker registry (Artifactory) on an internal network that uses a self signed certificate for authentication.

When Kubernetes starts up a

Answer 1

Kubernetes is likely using the docker daemon on the Kubernetes cluster nodes. For them to trust your local registry, you can the trusted registry hostnname to the file /etc/docker/daemon.json as follows:

{ "insecure-registries":["some.local.registry"] }

where some.local.registry is the hostname of the registry.

You need to restart the docker process(es) to make this effective. I did this for a domain that is not public and has no valid TLD, so I could not use cert-manager with letsencrypt.

You need to do the same on every machine that uses docker to connect to that registry.

Answer 2

You can access the keys for private docker registries in $HOME/.dockercfg or $HOME/.docker/config.json . If you add it to one of these search paths kubelet should use it as a credential when pulling the images.

• {--root-dir:-/var/lib/kubelet}/config.json
• {cwd of kubelet}/config.json
• ${HOME}/.docker/config.json
• /.docker/config.json
• {--root-dir:-/var/lib/kubelet}/.dockercfg
• {cwd of kubelet}/.dockercfg
• ${HOME}/.dockercfg
• /.dockercfg

https://kubernetes.io/docs/concepts/containers/images/#using-a-private-registry

The "Configuring Nodes to Authenticate to a Private Registry" section gives you a step by step on how to do it.

Answer 3

You basically have to tell the Docker daemon to trust your self-signed certificate by telling it to trust the Certificate Authority (CA) that you used to sign the certificate. You can find more information here on the section that says "Use self-signed certificates".

In particular for example for Linux:

Linux: Copy the domain.crt file to /etc/docker/certs.d/myregistrydomain.com:5000/ca.crt on every Docker host. You do not need to restart Docker.

This all different from authenticating by specifying ImagePullSecrets on your pods or docker login credentials in your docker config files.

Answer 4

The simplest solution I found after an extensive search is suggested in this guide by CoreOS : https://github.com/coreos/tectonic-docs/blob/master/Documentation/admin/add-registry-cert.md

It consists to create a secret that contains your certificate and a DaemonSet to populate it to /etc/docker/certs.d/my-private-insecure-registry.com/ca.crt on all the nodes of your cluster.

I think this answers your question because, when adding a new node, the DaemonSet is automatically executed on it.

I give the detailed solution below but all the credits goes to Kyle Brown (kbrwn) for his very cool guide (cf. link above).

Detailed solution

Lets suppose that your certificate is a file named ca.crt in your working directory. Create a secret from this file content :

kubectl create secret generic registry-ca --namespace kube-system --from-file=registry-ca=./ca.crt

Then, use the following DaemonSet that mounts the certificate as the file /home/core/registry-ca and copy it to the desired location : /etc/docker/certs.d/reg.example.com/ca.crt.

Simply replace my-private-insecure-registry.com with the hostname of your container registry.

apiVersion: extensions/v1beta1
kind: DaemonSet
metadata:
  name: registry-ca
  namespace: kube-system
  labels:
    k8s-app: registry-ca
spec:
  template:
    metadata:
      labels:
        name: registry-ca
    spec:
      containers:
      - name: registry-ca
        image: busybox
        command: [ 'sh' ]
        args: [ '-c', 'cp /home/core/registry-ca /etc/docker/certs.d/my-private-insecure-registry.com/ca.crt && exec tail -f /dev/null' ]
        volumeMounts:
        - name: etc-docker
          mountPath: /etc/docker/certs.d/my-private-insecure-registry.com
        - name: ca-cert
          mountPath: /home/core
      terminationGracePeriodSeconds: 30
      volumes:
      - name: etc-docker
        hostPath:
          path: /etc/docker/certs.d/my-private-insecure-registry.com
      - name: ca-cert
        secret:
          secretName: registry-ca

Save the file as registry-ca-ds.yaml and then create the DaemonSet :

kubectl create -f registry-ca-ds.yaml

You can now check that your application correctly pulls from your private self-signed registry.

As mentioned, the certificate will be added to new nodes' docker in an automatic fashion by the registry-ca DaemonSet. If you want to avoid this, simply delete the DaemonSet :

kubectl delete ds registry-ca --namespace kube-system

I think this is more secure than setting the insecure-registries flag of the docker daemon. Also, it is resilient to new nodes.