Scripted jenkinsfile parallel stage

Question

I am attempting to write a scripted Jenkinsfile using the groovy DSL which will have parallel steps within a set of stages.

Here is my jenkinsfile:

n         


        

Answer 1

I have used stage{} in parallel blocks several times. Then each stage shows up in the Stage view. The parent stage that contains parallel doesn't include the timing for all the parallel stages, but each parallel stage shows up in stage view.

In blue ocean, the parallel stages appear separately instead of the stages showing. If there is a parent stage, it shows as the parent of the parallel stages.

If you don't have the same experience, maybe a plugin upgrade is due.

Answer 2

To simplify the answer of @Ed Randall here. Remember this is Jenkinsfile scripted (not declarative)

stage("Some Stage") {
    // Stuff ...
}


stage("Parallel Work Stage") {

    // Prealocate dict/map of branchstages
    def branchedStages = [:]

    // Loop through all parallel branched stage names
    for (STAGE_NAME in ["Branch_1", "Branch_2", "Branch_3"]) {

        // Define and add to stages dict/map of parallel branch stages
        branchedStages["${STAGE_NAME}"] = {
            stage("Parallel Branch Stage: ${STAGE_NAME}") {
                // Parallel stage work here
                sh "sleep 10"
            }
        }

    }

    // Execute the stages in parallel
    parallel branchedStages
}


stage("Some Other Stage") {
    // Other stuff ...
}

Please pay attention to the curly braces. This will result in the following result (with the BlueOcean Jenkins Plugin):

Answer 3

Here's a simple example without loops or functions based on @Ed Randall's post:

node('docker') {
    stage('unit test') {
        parallel([
            hello: {
                echo "hello"
            },
            world: {
                echo "world"
            }
        ])
    }

    stage('build') {
        def stages = [:]

        stages["mac"] = {
            echo "build for mac"
        }
        stages["linux"] = {
            echo "build for linux"
        }

        parallel(stages)
    }
}

...which yields this:

Note that the values of the Map don't need to be stages. You can give the steps directly.

Answer 4

In Jenkins scripted pipeline, parallel(...) takes a Map describing each stage to be built. Therefore you can programatically construct your build stages up-front, a pattern which allows flexible serial/parallel switching.
I've used code similar to this where the prepareBuildStages returns a List of Maps, each List element is executed in sequence whilst the Map describes the parallel stages at that point.

// main script block
// could use eg. params.parallel build parameter to choose parallel/serial 
def runParallel = true
def buildStages

node('master') {
  stage('Initialise') {
    // Set up List<Map<String,Closure>> describing the builds
    buildStages = prepareBuildStages()
    println("Initialised pipeline.")
  }

  for (builds in buildStages) {
    if (runParallel) {
      parallel(builds)
    } else {
      // run serially (nb. Map is unordered! )
      for (build in builds.values()) {
        build.call()
      }
    }
  }

  stage('Finish') {
      println('Build complete.')
  }
}

// Create List of build stages to suit
def prepareBuildStages() {
  def buildStagesList = []

  for (i=1; i<5; i++) {
    def buildParallelMap = [:]
    for (name in [ 'one', 'two', 'three' ] ) {
      def n = "${name} ${i}"
      buildParallelMap.put(n, prepareOneBuildStage(n))
    }
    buildStagesList.add(buildParallelMap)
  }
  return buildStagesList
}

def prepareOneBuildStage(String name) {
  return {
    stage("Build stage:${name}") {
      println("Building ${name}")
      sh(script:'sleep 5', returnStatus:true)
    }
  }
}

The resulting pipeline appears as:

There are certain restrictions on what can be nested within a parallel block, refer to the pipeline documentation for exact details. Unfortunately much of the reference seems biased towards declarative pipeline, despite it being rather less flexible than scripted (IMHO). The pipeline examples page was the most helpful.

Answer 5

I was also trying similar sort of steps to execute parallel stages and display all of them in a stage view. You should write a stage inside a parallel step as shown in the following code block.

// Jenkinsfile (Scripted Pipeline)

stage('Build') {
    /* .. Your code/scripts .. */
}

stage('Test') {
    parallel 'linux': {
        stage('Linux') {
            /* .. Your code/scripts .. */
        }
    }, 'windows': {
        stage('Windows') {
            /* .. Your code/scripts .. */
        }
    }
}

Answer 6

Here is an example from their docs:

Parallel execution

The example in the section above runs tests across two different platforms in a linear series. In practice, if the make check execution takes 30 minutes to complete, the "Test" stage would now take 60 minutes to complete!

Fortunately, Pipeline has built-in functionality for executing portions of Scripted Pipeline in parallel, implemented in the aptly named parallel step.

Refactoring the example above to use the parallel step:

// Jenkinsfile (Scripted Pipeline)


stage('Build') {
    /* .. snip .. */
}

stage('Test') {
    parallel linux: {
        node('linux') {
            checkout scm
            try {
                unstash 'app'
                sh 'make check'
            }
            finally {
                junit '**/target/*.xml'
            }
        }
    },
    windows: {
        node('windows') {
            /* .. snip .. */
        }
    }
}