Gitlabs artifact of one project used in further projects

Question

Question

• What is the best way to carry artifacts (jar, class, war) among projects when using docker containers in CI phase.

Let me explain my

Answer 1

Hello you must take a look at a script named get-last-successful-build-artifact.sh and developed by morph027.

https://gitlab.com/morph027/gitlab-ci-helpers

This script allow to download an artifact and unzip it in the project root. It use Gitlab API to retrieve latest successful build and download corresponding artifact. You can combine multiple artifacts and unzip wherever you want just by updating the script a little.

I'm also currently starting a PHP library to handle build artifacts but it's in a very early stage and tied with laravel for the moment.

For the moment there is no easy way to handle artifact usage between projects, you must build your own using that tools.

I think using shell executor is not the right solution, it's very dangerous because you can't verify the file on the server used during the build !

Hope this help :)

Answer 2

As of this writing artifacts cannot be shared across project only within the pipeline. See https://docs.gitlab.com/ee/ci/yaml/README.html#artifacts

However there is an open feature to enable this facility which is not yet implemented. https://gitlab.com/gitlab-org/gitlab-ce/issues/14728

Answer 3

In GitLab silver and premium, there is the $CI_JOB_TOKEN available, which allows the following .gitlab-ci.yaml snippet:

build_submodule:
  image: debian
  stage: test
  script:
  - apt update && apt install -y unzip
  - curl --location --output artifacts.zip "https://gitlab.example.com/api/v4/projects/1/jobs/artifacts/master/download?job=test&job_token=$CI_JOB_TOKEN"
  - unzip artifacts.zip
  only:
  - tags

However, if you do not have silver or higher gitlab subscriptions, but rely on free tiers, it is also possible to use the API and pipeline triggers.

Let's assume we have project A building app.jar which is needed by project B.

First, you will need an API Token. Go to Profile settings/Access Tokens page to create one, then store it as a variable in project B. In my example it's GITLAB_API_TOKEN.

In the CI / CD settings of project B add a new trigger, for example "Project A built". This will give you a token which you can copy. Open project A's .gitlab-ci.yaml and copy the trigger_build: section from project B's CI / CD settings trigger section.

Project A:

trigger_build:
  stage: deploy
  script:
    - "curl -X POST -F token=TOKEN -F ref=REF_NAME https://gitlab.example.com/api/v4/projects/${PROJECT_B_ID}/trigger/pipeline"

Replace TOKEN with that token (better, store it as a variable in project A -- then you will need to make it token=${TRIGGER_TOKEN_PROJECT_B} or something), and replace REF_NAME with your branch (e.g. master).

Then, in project B, we can write a section which only builds on triggers and retrieves the artifacts.

Project B:

download:
  stage: deploy
  only:
    - triggers
  script:
    - "curl -O --header 'PRIVATE-TOKEN: ${GITLAB_API_TOKEN}' https://gitlab.example.com/api/v4/projects/${PROJECT_A_ID}/jobs/${REMOTE_JOB_ID}/artifacts/${REMOTE_FILENAME}"

If you know the artifact path, then you can replace ${REMOTE_FILENAME} with it, for example build/app.jar. The project ID can be found in the CI / CD settings.

I extended the script in project A to pass the remaining information as documented in the trigger settings section:

Add variables[VARIABLE]=VALUE to an API request. Variable values can be used to distinguish between triggered pipelines and normal pipelines.

So the trigger passes the REMOTE_JOB_ID and the REMOTE_FILENAME, but of course you can modify this as you need it:

curl -X POST \
     -F token=TOKEN \
     -F ref=REF_NAME \
     -F "variables[REMOTE_FILENAME]=build/app.jar" \
     -F "variables[REMOTE_JOB_ID]=${CI_JOB_ID}" \
     https://gitlab.example.com/api/v4/projects/${PROJECT_B_ID}/trigger/pipeline

Answer 4

Cool, found my snippet being referenced here ;)

is it possible to use get-last-successful-build-artifact.sh without private-token (in a world-readable repository)? e.g. to share an artifact download command w/o exposing your token

Yes, just add it as a secret variable in project settings -> pipelines -> secret variables.

Answer 5

arry artifacts (jar, class, war) among projects

That should be what the package Registry is for.

With GitLab 13.3 (August 2020), it is now available for free!

Package Registry now available in Core

A year and a half ago, we expanded our support for Java projects and developers by building Maven support directly into GitLab. Our goal was to provide a standardized way to share packages and have version control across projects.

Since then, we’ve invested to build out the Package team further while working with our customers and community to better understand your use cases. We also added support for Node, C#/.NET, C/C++, Python, PHP, and Go developers.

Your increased adoption, usage, and contributions to these features have allowed us to expand our vision to a more comprehensive solution, integrated into our single application, which supports package management for all commonly-used languages and binary formats.
This goal could not have been achieved without the explicit support of the GitLab community.

As part of GitLab’s stewardship promises, we are excited to announce that the basic functionality for each package manager format is now available in the GitLab Core Edition.
This means that if you use npm, Maven, NuGet, Conan, PyPI, Composer or Go modules, you will be able to:

• Use GitLab as a private (or public) package registry
• Authenticate using your GitLab credentials, personal access, or job token
• Publish packages to GitLab
• Install packages from GitLab
• Search for packages hosted on GitLab
• Access an easy-to-use UI that displays package details and metadata and allows you to download any relevant files
• Ensure that your contributions are available for ALL GitLab users

We look forward to hearing your feedback and continuing to improve these features with all of our users.

See Documentation and Issue.

See this video.