Docker image push over SSH (distributed)

Question

TL;DR Basically, I am looking for this:

docker push myimage ssh://myvps01.vpsprovider.net/

I am failing to grasp the rationale

Answer 1

I made a command line utility just for this scenario.

It sets up a temporary private docker registry on the server, establishes an SSH Tunnel from your localhost, pushes your image, then cleans up after itself.

The benefit of this approach over docker save is that only the new layers are pushed to the server, resulting in a quicker upload.

Oftentimes using an intermediate registry like dockerhub is undesirable, and cumbersome.

https://github.com/brthor/docker-push-ssh

Install:

pip install docker-push-ssh

Example:

docker-push-ssh -i ~/my_ssh_key username@myserver.com my-docker-image

Biggest caveat is that you have to manually add your local ip to docker's insecure_registries config.

https://stackoverflow.com/questions/32808215/where-to-set-the-insecure-registry-flag-on-mac-os

Answer 2

Saving/loading an image on to a Docker host and pushing to a registry (private or Hub) are two different things.

The former @Thomasleveil has already addressed.

The latter actually does have the "smarts" to only push required layers.

You can easily test this yourself with a private registry and a couple of derived images.

If we have two images and one is derived from the other, then doing:

docker tag baseimage myregistry:5000/baseimage
docker push myregistry:5000/baseimage

will push all layers that aren't already found in the registry. However, when you then push the derived image next:

docker tag derivedimage myregistry:5000/derivedimage
docker push myregistry:5000/derivedimage

you may noticed that only a single layer gets pushed - provided your Dockerfile was built such that it only required one layer (e.g. chaining of RUN parameters, as per Dockerfile Best Practises).

On your Docker host, you can also run a Dockerised private registry.

See Containerized Docker registry

To the best of my knowledge and as of the time of writing this, the registry push/pull/query mechanism does not support SSH, but only HTTP/HTTPS. That's unlike Git and friends.

See Insecure Registry on how to run a private registry through HTTP, especially be aware that you need to change the Docker engine options and restart it:

Open the /etc/default/docker file or /etc/sysconfig/docker for editing.

Depending on your operating system, your Engine daemon start options.

Edit (or add) the DOCKER_OPTS line and add the --insecure-registry flag.

This flag takes the URL of your registry, for example.

DOCKER_OPTS="--insecure-registry myregistrydomain.com:5000"

Close and save the configuration file.

Restart your Docker daemon

You will also find instruction to use self-signed certificates, allowing you to use HTTPS.

Using self-signed certificates

[...]

This is more secure than the insecure registry solution. You must configure every docker daemon that wants to access your registry

Generate your own certificate:

mkdir -p certs && openssl req \ -newkey rsa:4096 -nodes -sha256 -keyout certs/domain.key \ -x509 -days 365 -out certs/domain.crt

Be sure to use the name myregistrydomain.com as a CN.

Use the result to start your registry with TLS enabled

Instruct every docker daemon to trust that certificate.

This is done by copying the domain.crt file to /etc/docker/certs.d/myregistrydomain.com:5000/ca.crt.

Don’t forget to restart the Engine daemon.

Answer 3

If you want to push docker images to a given host, there is already everything in Docker to allow this. The following example shows how to push a docker image through ssh:

docker save <my_image> | ssh -C user@my.remote.host.com docker load

• docker save will produce a tar archive of one of your docker images (including its layers)
• -C is for ssh to compress the data stream
• docker load creates a docker image from a tar archive

---

Note that the combination of a docker registry + docker pull command has the advantage of only downloading missing layers. So if you frequently update a docker image (adding new layers, or modifying a few last layers) then the docker pull command would generate less network traffic than pushing complete docker images through ssh.