GitLab CI preserve environment between build stages

Question

I am working on a python project and using miniconda to manage my environment. I am using GitLab for CI with the following runner configuration

stages:
  - b         


        

Answer 1

The independence of the jobs is a design feature. You might have noticed that GitLab's interface allows you to re-run a single job which wouldn't be possible if the jobs depended on each other.

I don't know what Miniconda exactly performs but if it builds a virtual environment in specific folders, you can use cache to preserve the content of those folders between the jobs. However, you cannot fully rely on it because the documentation states that...

The cache is provided on a best-effort basis, so don't expect that the cache will be always present. For implementation details, please check GitLab Runner.

Considering that your job absolutely depends on the environment being built, you would need a mechanism to detect whether the (cached) environment exists and re-create it only if needed.

I think you are taking good path trying to separate the environment setup and the jobs because it might save lots of time in case you decide one day to run different tests simultaneously (jobs at the same stage run in parallel).