Is there a way to setup bootstrap actions to run on EMR after core services are installed (Spark etc)?

问题

Is there a way to setup bootstrap actions to run on EMR after core services are installed (Spark etc)? I am using emr-5.27.0.

回答1:

You can submit some script as a step, not a bootstrap. For example, I made an SSL certificate update script and it is applied to the EMR by a step. This is a part of my lambda function in Python language. But you can add this step by manually on the console, or other languages.

Steps=[{
    'Name': 'PrestoCertificate',
    'ActionOnFailure': 'CONTINUE',
    'HadoopJarStep': {
        'Jar': 's3://ap-northeast-2.elasticmapreduce/libs/script-runner/script-runner.jar',
        'Args': ['s3://myS3/PrestoSteps_InstallCertificate.sh']
    }
}]

The key point is script-runner.jar that is pre-built by amazon and you can use that for each region by changing the region prefix. It receives a .sh file and runs it.

One thing you should know is, the script will run on all the nodes and if you want to do it only the master instance then you have to use if-else statement.

#!/bin/bash
BOOL=`cat /emr/instance-controller/lib/info/instance.json | jq .isMaster`

if [ $BOOL == "true" ]
then
    <your code>
fi

来源：https://stackoverflow.com/questions/58211309/is-there-a-way-to-setup-bootstrap-actions-to-run-on-emr-after-core-services-are