Finding the layers and layer sizes for each Docker image

Question

For research purposes I\'m trying to crawl the public Docker registry ( https://registry.hub.docker.com/ ) and find out 1) how many layers an average image has and 2) the si

Answer 1

Check out dive written in golang.

Awesome tool!

Answer 2

I've solved this problem by using the search function on Docker's website where '*' is a valid search that returns 200k repositories and then I crawled each invididual page. HTML parsing allows me to extract all the image names on each page.

Answer 3

They have a very good answer here: https://stackoverflow.com/a/32455275/165865

Just run below images:

docker run --rm -v /var/run/docker.sock:/var/run/docker.sock nate/dockviz images -t

Answer 4

one more tool : https://github.com/CenturyLinkLabs/dockerfile-from-image

GUI using ImageLayers.io

Answer 5

It's indeed doable to query the manifest or blob info from docker registry server without pulling the image to local disk.

You can refer to the Registry v2 API to fetch the manifest of image.

GET /v2/<name>/manifests/<reference>

Note, you have to handle different manifest version. For v2 you can directly get the size of layer and digest of blob. For v1 manifest, you can HEAD the blob download url to get the actual layer size.

There is a simple script for handling above cases that will be continuously maintained.

Answer 6

1. https://hub.docker.com/search?q=* shows all the images in the entire Docker hub, it's not possible to get this via the search command as it doesnt accept wildcards.

2. As of v1.10 you can find all the layers in an image by pulling it and using these commands:

   docker pull ubuntu
   ID=$(sudo docker inspect -f {{.Id}} ubuntu)
   jq .rootfs.diff_ids /var/lib/docker/image/aufs/imagedb/content/$(echo $ID|tr ':' '/')
   

3) The size can be found in /var/lib/docker/image/aufs/layerdb/sha256/{LAYERID}/size although LAYERID != the diff_ids found with the previous command. For this you need to look at /var/lib/docker/image/aufs/layerdb/sha256/{LAYERID}/diff and compare with the previous command output to properly match the correct diff_id and size.