AWS Pass in variable into buildspec.yml from CodePipeline

Question

I have an AWS CodePipeline that invokes CodeBuild in the Build Stage.

The question is how do I pass in an environment variable from CodePipeline that can be read in the CodeBuild's buildspec.yml?

I know I can set environment variables in CodeBuild, but I want to use the same CodeBuild project for dev, qa, and prod environments. I don't see how I can pass an environment variable from CodePipeline that makes it all the way to the buildspec.yml

Example buildspec.yml

version: 0.1

phases:   
  build:
    commands:
      - npm install
      - npm build -- --env ${CURRENT_ENVIRONMENT}

Where CURRENT_ENVIRONMENT would be the variable I set in the CodePipeline Stage action.

Answer 1

This feature isn't available today.

A workaround would be to create a different CodeBuild project for each stage with different environment variables.

You can find details on using an environment variable in your builspec.yml commands here: http://docs.aws.amazon.com/codebuild/latest/userguide/build-spec-ref.html

Answer 2

You can actually pass environment variables in CodeBuild cloudformation as below:

Build:
    Type: AWS::CodeBuild::Project
    Properties:
      Name:
        !Sub Build-${AWS::StackName}
      Description: Build your project
      Environment:
        Type: LINUX_CONTAINER
        ComputeType: BUILD_GENERAL1_SMALL
        Image: node8
        EnvironmentVariables:
          - Name: CURRENT_ENVIRONMENT
            Type: PLAINTEXT
            Value: staging

And in your buildspec.yml you can reference the environment like this,

version: 0.2

phases:
  install:
    commands:
      - npm install

  build:
    commands:
      - npm build -- --env ${CURRENT_ENVIRONMENT}

Answer 3

If you don't want to use CF, you can set ENV vars on your CodeBuild project in the AWS UI.

In AWS, go to your code builder project, in the top right corner click "edit" and select "environment." On the edit environment page, click the "Additional Configuration" drop down. In there you will see inputs for "name" and "value." "Name" is where you set your ENV and "value" is where you set your value for that variable.

Example: set API_ENV in "name" and development in "value." Then, in your buildspec.yml, you can use $API_ENV.

Answer 4

You can use build environment variables to conditionally pass from buildspec to npm build by detecting which CodeBuild job or CodePipeline job is running. If you have one CodePipeline listening to /dev commits and another listening to /master commits, for example, this works perfectly.

Here's an example which runs a different PROD vs DEV build:

  build:
    commands:
   - |
     if expr "${CODEBUILD_BUILD_ARN}" : ".*build/MyProjectDev-" >/dev/null; then
       yarn run build-dev;
     fi
   - |
     if expr "${CODEBUILD_BUILD_ARN}" : ".*build/MyProject-" >/dev/null; then
       yarn run build-prod;
     fi

Answer 5

I've created a lambda function that updates an existing codebuild project's environment variables. You can the start the build (codebuild.start) after updating the variables. Looks something like this (nodejs):

var params = {
    "name": "Build-project-name",
    "description": "Build this project.",

    "environment": {
        "type": "LINUX_CONTAINER",
        "image": "aws/codebuild/standard:1.0",
        "computeType": "BUILD_GENERAL1_LARGE",
        "environmentVariables": [
        {
            "name": "MY_ENVIRONMENT_VARIABLE",
            "value": "VALUE_OF_ENVIRONMENT_VARIABLE",
            "type": "PLAINTEXT"
        }]
    }
}

codebuild.updateProject(params, function(err, data) {
    if (err) {
        console.log(err, err.stack); // an error occurred
    }
    else     {
        console.log(data);           // successful response
    }
});

Answer 6

The CodeBuild initiator is an environment variable in CodeBuild and this can be used to read in the CodePipeline name.

So if you include your environment in your CodePipeline name as a suffix (-dev or -prod, for instance), then you can parse it out like so.

version: 0.2
phases:
  build:
    commands:
      - CURRENT_ENVIRONMENT=`echo $CODEBUILD_INITIATOR | cut -d '-' -f2 | tr '[:upper:]' '[:lower:]'`
      - echo "My env is $CURRENT_ENVIRONMENT"

Answer 7

I created a small python script to parse the $CODEBUILD_INITIATOR variable that is passed. Below is the buildspec.yml and below that is the python script that I include with the build and call.

  build:
    commands:
      - |
        PIPELINE_ENV=$(python3 codebuild_env_parser.py $CODEBUILD_INITIATOR)
        OUTPUT_STATUS=$?
        if [ "$OUTPUT_STATUS" = "0" ]; then
          echo "Success finding a valid environment from codebuild_env_parser.py."
        else
          echo "Failure finding a valid environment from codebuild_env_parser.py. Check the script to see if the codepipeline env was passed correctly."
        fi

Python script (codebuild_env_parser.py):

import sys


def main():
    args = sys.argv
    if len(args) == 2:
        env_list = ["dev", "prod"]
        pipeline_invoker = args[1].lower()
        code_pipeline_name = pipeline_invoker.split("codepipeline/")[1]
        env_name = code_pipeline_name.split("-project-name")[0]
        if env_name in env_list:
            print("{}".format(env_name))
            sys.exit(0)
        else:
            sys.exit(1)
    else:
        sys.exit(1)


main()

You'll have to tweak some variable values here if you want this to work. Namely, "-project-name".

Answer 8

As of today, you can set environment variables for CodeBuild build jobs in your pipeline. This improvement makes it possible to reuse the same build project for multiple actions and simplify deployments to staging and production environments in your pipeline.

name: Build
actions:
  - name: MyBuildJob
    actionTypeId:
      category: Build
      owner: AWS
      provider: CodeBuild
      version: '1'
    runOrder: 1
    configuration:
      ProjectName: my-build-project
      PrimarySource: MyApplicationSource1
      EnvironmentVariables: '[{"name":"CURRENT_ENVIRONMENT}","value":"Production","type":"PLAINTEXT"},
                              {"name":"UseParamStore","value":"CURRENT_ENVIRONMENT}","type":"PARAMETER_STORE"}]'

https://docs.aws.amazon.com/codepipeline/latest/userguide/action-reference-CodeBuild.html#action-reference-CodeBuild-config