How to pass environment variables when programmatically starting a new Amazon EC2 from image?

Question

I am using AWS Java API RunInstance() to start a new EC2 instance from my custom AMI image. How do I pass environment variables to the new EC2 INSTANCE such as database url,

Answer 1

DISCLAIMER: I am not a sys admin!

I use a secure S3 bucket meaning a bucket that only the instance you're launching has access to. You can setup an IAM role that looks like:

{
  "Version": "2012-10-17",
  "Statement": [
    {
      "Effect": "Allow",
      "Action": [
        "s3:Get*",
        "s3:List*"
      ],
      "Resource": "arn:aws:s3:::some-secure-bucket/*"
    }
  ]
}

Then you can upload your .env file in that bucket (store it encrypted). Then to access it on your EC2 instance, you could use the AWS cli tools:

sudo apt-get install -y python-pip (for aws s3 CLI library)
sudo pip install awscli
aws s3 cp --region us-east-1 s3://some-secure-bucket/.some-dot-env-file output_file_path

You can pull this file down when the code runs or optionally make it happen at boot by putting the aforementioned cp command in an init script located somewhere like /etc/init.d/download_credentials.sh

I think this is a really good option for downloading things that every instance using an AMI needs like credentials. However, if you want to specify per instance metadata, I just implemented using tags which I think works nice. To do this, alter the above IAM role with something more like:

{
  "Version": "2012-10-17",
  "Statement": [
    {
      "Effect": "Allow",
      "Action": [
        "s3:Get*",
        "s3:List*"
      ],
      "Resource": "arn:aws:s3:::some-secure-bucket/*"
    },
    {
      "Effect": "Allow",
      "Action": [
        "ec2:DescribeInstances",
        "ec2:DescribeTags"
      ],
      "Resource": "*"
    }
  ]
}

Then install ec2-api-tools

sudo sed -i.dist 's,universe$,universe multiverse,' /etc/apt/sources.list
sudo apt-get update
sudo apt-get install -y ec2-api-tools

And now you should be able to get per instance metadata through tags, such as the "Name" of your instance:

ec2-describe-tags --filter resource-id="$(ec2metadata --instance-id)" --filter "key=Name" | cut -f5

Note: I suck at bash so I'm stripping the name in ruby but you could use tr to remove the newline if you're into it!

Answer 2

http://alestic.com/2009/06/ec2-user-data-scripts explains how to do this with user-data. for gotchas about using Java see AmazonEC2 launch with userdata.

note that I've seen mention that this doesn't work with Windows, only Unix.

[update] more data on setting environment variables here: https://forums.aws.amazon.com/message.jspa?messageID=139744

[after much testing] for me, echoing the environment variables into /etc/environment works best, like this:

 reservation = connection.run_instances(image_id = image_id,
  key_name = keypair,
  instance_type = 'm1.small',
  security_groups = ['default'],
  user_data = '''#!/bin/sh\necho export foozle=barzle >> /etc/environment\n''')

then upon login:

ubuntu@ip-10-190-81-29:~$ echo $foozle
barzle

Answer 3

You can also use instance metadata retrieval as explained at https://docs.aws.amazon.com/AWSEC2/latest/UserGuide/instancedata-data-retrieval.html

From the above document, the following GET request would retrieve user data for an instance if you run it from within the instance:

GET http://169.254.169.254/latest/user-data

This way, user data can be retrieved dynamically even after the instance is already started and running.