How to enable / setup Dependency Caches for apt-get on BitBucket Pipelines

Question

I am using the following code in my bitbucket-pipelines.yml files to remotely deply code to a staging server.

image: php:7.1.1

pipelines:
  default:
    - step:
        script:
          # install ssh
          - apt-get update && apt-get install -y openssh-client
          # get the latest code
          - ssh user@domain.com -F ~/.ssh/config "cd /path/to/code && git pull"
          # update composer
          - ssh user@domain.com -F ~/.ssh/config "cd /path/to/code && composer update --no-scripts"
          # optimise files
          - ssh user@domain.com -F ~/.ssh/config "cd /path/to/code && php artisan optimize"

This all works, except that each time the pipeline is run, the ssh client is downloaded and installed everything (adding ~30 seconds to the build time). Is there way I can cache this step?

And how can I go about caching the apt-get step?

For example, would something like this work (or what changes are needed to make the following work):

pipelines:
  default:
    - step:
        caches:
          - aptget
        script:
          - apt-get update && apt-get install -y openssh-client

definitions:
  caches:
    aptget: which ssh

Answer 1

This is a typical scenario where you should use your own Docker image instead of one of the ones provided by Atlassian. (Or search for a Docker image which provides exactly this.)

In your simple case, this Dockerfile should be enough:

FROM php:7.1.1

RUN apt-get update && \
    apt-get install -y openssh-client

Then, create a DockerHub account, publish the image and reference it in bitbucket-pipelines.yml.

Answer 2

Unfortunately, the parts that take the time are unsafe or pointless to cache. Remember that the pipeline caches may be deleted at any time, so you always need to run the commands anyway.

apt-get update doesn't use a cache, so will download the latest indexes every time.

apt-get install caches downloaded packages in /var/cache/apt so you could save that. However this probably won't actually save any time

Fetched 907 kB in 0s (998 kB/s)

The actual installed packages cannot be cached, because they a) are spread around multiple shared files and directories and b) may not be portable to different docker images.

At a deeper level, satisfactory interaction between caching, apt-get update, and Docker is a complex issue.