When running Docker for a long time, there are a lot of images in system. How can I remove all unused Docker images at once safety to free up the storage?
In additio
There is sparrow plugin docker-remove-dangling-images you can use to clean up stopped containers and unused (dangling) images:
$ sparrow plg run docker-remove-dangling-images
It works both for Linux and Windows OS.
Occasionally I have run into issues where Docker will allocate and continue to use disk space, even when the space is not allocated to any particular image or existing container. The latest way I generated this issue accidentally was using "docker-engine" centos build instead of "docker" in RHEL 7.1. What seems to happen is sometimes the container clean-ups are not completed successfully and then the space is never reused. When the 80GB drive I allocated as /
was filled with /var/lib/docker
files I had to come up with a creative way to resolve the issue.
Here is what I came up with. First to resolve the disk full error:
Stop docker: systemctl stop docker
Allocated a new drive mounted as say /mnt/docker
.
Move all the files in /var/lib/docker
to /mnt/docker
. I used the command:
rsync -aPHSx --remove-source-files /var/lib/docker/ /mnt/docker/
Mount the new drive to /var/lib/docker
.
At this point I no longer had a disk full error, but I was still wasting a huge amount of space. The next steps are to take care of that.
Start Docker: systemctl start docker
Save the all the images:
docker save $(docker images |sed -e '/^<none>/d' -e '/^REPOSITORY/d' -e 's,[ ][ ]*,:,' -e 's,[ ].*,,') > /root/docker.img
Uninstall docker.
Erase everything in /var/lib/docker
:
rm -rf /var/lib/docker/[cdintv]*
Reinstall docker
Enable docker: systemctl enable docker
Start docker: systemctl start docker
Restore images:
docker load < /root/docker.img
Start any persistent containers you need running.
This dropped my disk usage from 67 GB for docker to 6 GB for docker.
I do not recommend this for everyday use. But it is useful to run when it looks like docker has lost track of used disk space do to software errors, or unexpected reboots.
The following command will delete images older than 48 hours.
$ docker image prune --all --filter until=48h
To remove old tagged images that are more than a month old:
$ docker images --no-trunc --format '{{.ID}} {{.CreatedSince}}' \
| grep ' months' | awk '{ print $1 }' \
| xargs --no-run-if-empty docker rmi
Note that it'll fail to remove images that are used by a container, referenced in a repository, has dependent child images... which is probably what you want. Else just add -f
flag.
Example of /etc/cron.daily/docker-gc
script:
#!/bin/sh -e
# Delete all stopped containers (including data-only containers).
docker ps -a -q --no-trunc --filter "status=exited" | xargs --no-run-if-empty docker rm -v
# Delete all tagged images more than a month old
# (will fail to remove images still used).
docker images --no-trunc --format '{{.ID}} {{.CreatedSince}}' | grep ' months' | awk '{ print $1 }' | xargs --no-run-if-empty docker rmi || true
# Delete all 'untagged/dangling' (<none>) images
# Those are used for Docker caching mechanism.
docker images -q --no-trunc --filter dangling=true | xargs --no-run-if-empty docker rmi
# Delete all dangling volumes.
docker volume ls -qf dangling=true | xargs --no-run-if-empty docker volume rm
I recently wrote a script to solve this on one of my servers:
#!/bin/bash
# Remove all the dangling images
DANGLING_IMAGES=$(docker images -qf "dangling=true")
if [[ -n $DANGLING_IMAGES ]]; then
docker rmi "$DANGLING_IMAGES"
fi
# Get all the images currently in use
USED_IMAGES=($( \
docker ps -a --format '{{.Image}}' | \
sort -u | \
uniq | \
awk -F ':' '$2{print $1":"$2}!$2{print $1":latest"}' \
))
# Get all the images currently available
ALL_IMAGES=($( \
docker images --format '{{.Repository}}:{{.Tag}}' | \
sort -u \
))
# Remove the unused images
for i in "${ALL_IMAGES[@]}"; do
UNUSED=true
for j in "${USED_IMAGES[@]}"; do
if [[ "$i" == "$j" ]]; then
UNUSED=false
fi
done
if [[ "$UNUSED" == true ]]; then
docker rmi "$i"
fi
done
Here is a script to clean up Docker images and reclaim the space.
#!/bin/bash -x
## Removing stopped container
docker ps -a | grep Exited | awk '{print $1}' | xargs docker rm
## If you do not want to remove all container you can have filter for days and weeks old like below
#docker ps -a | grep Exited | grep "days ago" | awk '{print $1}' | xargs docker rm
#docker ps -a | grep Exited | grep "weeks ago" | awk '{print $1}' | xargs docker rm
## Removing Dangling images
## There are the layers images which are being created during building a Docker image. This is a great way to recover the spaces used by old and unused layers.
docker rmi $(docker images -f "dangling=true" -q)
## Removing images of perticular pattern For example
## Here I am removing images which has a SNAPSHOT with it.
docker rmi $(docker images | grep SNAPSHOT | awk '{print $3}')
## Removing weeks old images
docker images | grep "weeks ago" | awk '{print $3}' | xargs docker rmi
## Similarly you can remove days, months old images too.
Original script
https://github.com/vishalvsh1/docker-image-cleanup
Usually Docker keeps all temporary files related to image building and layers at
/var/lib/docker
This path is local to the system, usually at THE root partition, "/".
You can mount a bigger disk space and move the content of /var/lib/docker
to the new mount location and make a symbolic link.
This way, even if Docker images occupy space, it will not affect your system as it will be using some other mount location.
Original post: Manage Docker images on local disk