Cloning a private Repo from gitlab CI job using HTTPS without exposing my credentials into CLI

问题

I have a gitlab ci job that does some work for me but it depends on another repo so I need to clone another repo inside this job. I can't use https cloning because gitlab will ask me for user name and password and the gitlab ci is non-interactive.
So how to clone gitlab repo using https inside gitlab ci job.
Thanks in advance.

回答1:

You should clone that repo using SSH and a read-only deploy key, possibly storing the token in a masked variable.

回答2:

I have found a way to authenticate my cloning. First, I navigate to my gitlab account setting then navigate to access tokens tab. Then create new access token with any name and you can leave expiring date empty (but for security, I have set an expiring date) then choose the right scope you want. In my case, I have read the repo scope.

Then I used the generated access token to clone my repo using https as follow

https://oauth2:YOUR_ACCESS_TOKEN@gitlab.com/PATH/TO/YOUR/REPO.git

To prevent CLI from exposing your access token you have to add your access token in CI variables list from gitlab as masked. SO when cloning is executed your access token will not be displayed. And then will access it as follow

https://oauth2:$ACCESS_TOKEN@gitlab.com/PATH/TO/YOUR/REPO.git

回答3:

If you are running gitlab version 8.12 or later, the permissions model was reworked. Along with this new permission model comes the the CI environment variable CI_JOB_TOKEN. The premium version of GitLab uses this environment variable for triggers, but you can use it to clone repos.

dummy_stage:
  script:
    - git clone https://gitlab-ci-token:${CI_JOB_TOKEN}@gitlab.instance/group/project.git

回答4:

I did not find the answer to my question after searching Google & stackoverflow for a while so I would like to share my solution here.

Context

I have a repo that contains some scripts that clone external repositories. For some reasons, the only way to authenticate with these repositories is HTTPS.

During the GitLab CI jobs, the git clones fail because git prompts about username & password.

Objective

I would like git to 'log in' with my credentials once and stop prompting for credentials at every git clone statement (like Docker does).

Solution

Let's assume I set some service credentials in GitLab CI variables called SERVICE_USER and SERVICE_PASS. I add the following steps at the beginning of every job that needs to perform git operations.

Windows

git config --global credential.helper C:/git_creds.bat
Set-Content -Value "@echo off" -Path C:/git_creds.bat
Add-Content -Value "echo username=$SERVICE_USER" -Path C:/git_creds.bat
Add-Content -Value "echo password=$SERVICE_PASS" -Path C:/git_creds.bat

# to test it
git clone https://my-scm-provider.com/project.git

Linux

git config --global credential.helper "/bin/bash /git_creds.sh"
echo '#!/bin/bash' > /git_creds.sh
echo "sleep 1" >> /git_creds.sh
echo "echo username=$SERVICE_USER" >> /git_creds.sh
echo "echo password=$SERVICE_PASS" >> /git_creds.sh

# to test it
git clone https://my-scm-provider.com/project.git

I added a sleep 1 because I faced failing clones and read this answer.

Beware

Do this if the CI job is not performed in a Docker container.

The credentials are exposed in the git_creds at the root of the CI environment. I could afford doing such an operation because my CI is performed in containers. Since containers data is not persistent, the git_creds file is erased after every job. Regarding the password, I use a GitLab masked variable to hide it from the logs.

keywords: https git clone headless non-interactive credential helper

来源：https://stackoverflow.com/questions/57143313/cloning-a-private-repo-from-gitlab-ci-job-using-https-without-exposing-my-creden