“no space left on device” even after removing all containers

Question

While experimenting with Docker and Docker Compose I suddenly ran into \"no space left on device\" errors. I\'ve tried to remove everything using methods suggested in similar qu

Answer 1

For future reference, if you have removed all the containers you can also try docker system prune which will remove dangling images, containers and anything else.

Answer 2

This may not directly answer the question but it can be useful in general if the Dockerfile used to create the image is available.

Make sure in particular to limit the number of layers that will be generated, hence, when writing the Dockerfile, avoid doing this:

RUN apt-get update && sudo apt-get install -y package1 
RUN apt-get update && sudo apt-get install -y package2 
RUN apt-get update && sudo apt-get install -y package3 

and do this instead:

RUN apt-get update && sudo apt-get install -y \
    package1 \
    package2 \
    package3 

Doing so drastically reduced the size of the image as well as the inode usage, since less layers are generated. This helped address the issue in my case (where the inodes would all get used up).

Make sure to remove the potential intermediate image that was generated by the failed build to free up space docker rmi <IMAGE_ID>.

For more tips, you can check out this site about Optimizing Docker images.

Answer 3

The problem is that /var/lib/docker is on the / filesystem, which is running out of inodes. You can check this by running df -i /var/lib/docker

Since /home's filesystem has sufficient inodes and disk space, moving Docker's working directory there there should get it going again.

(Note that the this assumes there is nothing valuable in the current Docker install.)

First stop the Docker daemon. On Ubuntu, run

sudo service docker stop

Then move the old /var/lib/docker out of the way:

sudo mv /var/lib/docker /var/lib/docker~

Now create a directory on /home:

sudo mkdir /home/docker

and set the required permissions:

sudo chmod 0711 /home/docker

Link the /var/lib/docker directory to the new working directory:

sudo ln -s /home/docker /var/lib/docker

Then restart the Docker daemon:

sudo service docker start

Then it should work again.