How to set a custom environment variable in EMR to be available for a spark Application

Question

I need to set a custom environment variable in EMR to be available when running a spark application.

I have tried adding this:

                   .         


        

Answer 1

Use classification yarn-env to pass environment variables to the worker nodes.

Use classification spark-env to pass environment variables to the driver, with deploy mode client. When using deploy mode cluster, use yarn-env.