Unable to pip install in Docker image as agent through Jenkins declarative pipeline

Question

I have yet another issue with permissions running Docker through Jenkins declarative pipeline. I want to build and publish a Python package through a Jenkins job in a D

Answer 1

You can try executing it as sudo:

 stage('Package') {
      steps {
        sh '''
            python -V
            sudo python -m pip install -r requirements.txt --user --no-cache
            sudo python setup.py sdist
           '''
      }
    }

You may have issues due Jenkins is not able to run command as sudo in that case I will recommend you to follow the steps mentioned in this article or in this SO question

Answer 2

I have found what I myself would think is the prettier solution:

stage("Python Test") {
  agent { 
    docker {
      label "docker && linux" 
      image "python:3.7"
    }
  }
  steps {
    withEnv(["HOME=${env.WORKSPACE}"]) {
      sh "pip install -r requirements.txt --user"
      # python stuff
    }
  }
  post {
    cleanup {
      cleanWs()
    }
  }
}

This workaround steers completely around the issue itself, installing the packages at user level. The issue here was that the HOME-directory was not initially writeable either, thus overwriting the HOME directory.

Answer 3

I had a very similar pipeline that I was running right after setting up Docker agents on my Jenkins system, so I thought my setup was wrong. Using the comments in your thread, I cooked up this solution:

First, you'll need to be root inside your container, so change your agent declaration to be similar to this:

agent {
    docker {
        image "python:3.7"
        args '--user 0:0'
    }
}

Now I was able to use pip install! However, subsequent runs of the job would try to run git clean and fail since the built files inside the container were created by root. To fix that, I ran the clean command inside the container as my last step:

steps {
    sh 'git clean -fdx'
}

Update:

I found a problem where a failed build wouldn't clean up and killed all of the builds after it. To fix this, I put the clean action as a post-build task that always runs:

post {
    cleanup {
        cleanWs()
    }
}