How do I make cloud-init startup scripts run every time my EC2 instance boots?

Question

I have an EC2 instance running an AMI based on the Amazon Linux AMI. Like all such AMIs, it supports the cloud-init system for running startup scripts based on the User Data

Answer 1

I struggled with this issue for almost two days, tried all of the solutions I could find and finally, combining several approaches, came up with the following:

MyResource:
  Type: AWS::EC2::Instance
  Metadata:
    AWS::CloudFormation::Init:
      configSets:
        setup_process:
          - "prepare"
          - "run_for_instance"
      prepare:
        commands:
          01_apt_update:
            command: "apt-get update"
          02_clone_project:
            command: "mkdir -p /replication && rm -rf /replication/* && git clone https://github.com/awslabs/dynamodb-cross-region-library.git /replication/dynamodb-cross-region-library/"
          03_build_project:
            command: "mvn install -DskipTests=true"
            cwd: "/replication/dynamodb-cross-region-library"
          04_prepare_for_apac:
            command: "mkdir -p /replication/replication-west && rm -rf /replication/replication-west/* && cp /replication/dynamodb-cross-region-library/target/dynamodb-cross-region-replication-1.2.1.jar /replication/replication-west/replication-runner.jar"
      run_for_instance:
        commands:
          01_run:
            command: !Sub "java -jar replication-runner.jar --sourceRegion us-east-1 --sourceTable ${TableName} --destinationRegion ap-southeast-1 --destinationTable ${TableName} --taskName -us-ap >/dev/null 2>&1 &"
            cwd: "/replication/replication-west"
  Properties:
    UserData:
      Fn::Base64:
        !Sub |
          #cloud-config
          cloud_final_modules:
           - [scripts-user, always]
          runcmd:
           - /usr/local/bin/cfn-init -v -c setup_process --stack ${AWS::StackName} --resource MyResource --region ${AWS::Region}
           - /usr/local/bin/cfn-signal -e $? --stack ${AWS::StackName} --resource MyResource --region ${AWS::Region}

This is the setup for DynamoDb cross-region replication process.

Answer 2

Another approach is to use #cloud-boothook in your user data script. From the docs:

Cloud Boothook

• Begins with #cloud-boothook or Content-Type: text/cloud-boothook.
• This content is boothook data. It is stored in a file under /var/lib/cloud and then executed immediately.
• This is the earliest "hook" available. There is no mechanism provided for running it only one time. The boothook must take care of this itself. It is provided with the instance ID in the environment variable INSTANCE_ID. Use this variable to provide a once-per-instance set of boothook data.

Answer 3

In 11.10, 12.04 and later, you can achieve this by making the 'scripts-user' run 'always'. In /etc/cloud/cloud.cfg you'll see something like:

cloud_final_modules:
 - rightscale_userdata
 - scripts-per-once
 - scripts-per-boot
 - scripts-per-instance
 - scripts-user
 - keys-to-console
 - phone-home
 - final-message

This can be modified after boot, or cloud-config data overriding this stanza can be inserted via user-data. Ie, in user-data you can provide:

#cloud-config
cloud_final_modules:
 - rightscale_userdata
 - scripts-per-once
 - scripts-per-boot
 - scripts-per-instance
 - [scripts-user, always]
 - keys-to-console
 - phone-home
 - final-message

That can also be '#included' as you've done in your description. Unfortunately, right now, you cannot modify the 'cloud_final_modules', but only override it. I hope to add the ability to modify config sections at some point.

There is a bit more information on this in the cloud-config doc at https://github.com/canonical/cloud-init/tree/master/doc/examples

Alternatively, you can put files in /var/lib/cloud/scripts/per-boot , and they'll be run by the 'scripts-per-boot' path.

Answer 4

One possibility, although somewhat hackish, is to delete the lock file that cloud-init uses to determine whether or not the user-script has already run. In my case (Amazon Linux AMI), this lock file is located in /var/lib/cloud/sem/ and is named user-scripts.i-7f3f1d11 (the hash part at the end changes every boot). Therefore, the following user-data script added to the end of the Include file will do the trick:

#!/bin/sh
rm /var/lib/cloud/sem/user-scripts.*

I'm not sure if this will have any adverse effects on anything else, but it has worked in my experiments.

Answer 5

please use the below script above your bash script.

example: here m printing hello world to my file

stop instance before adding to userdata

script

Content-Type: multipart/mixed; boundary="//"
MIME-Version: 1.0

--//
Content-Type: text/cloud-config; charset="us-ascii"
MIME-Version: 1.0
Content-Transfer-Encoding: 7bit
Content-Disposition: attachment; filename="cloud-config.txt"

#cloud-config
cloud_final_modules:
- [scripts-user, always]

--//
Content-Type: text/x-shellscript; charset="us-ascii"
MIME-Version: 1.0
Content-Transfer-Encoding: 7bit
Content-Disposition: attachment; filename="userdata.txt"

#!/bin/bash
/bin/echo "Hello World." >> /var/tmp/sdksdfjsdlf
--//

Answer 6

cloud-init supports this now natively, see runcmd vs bootcmd command descriptions in the documentation (http://cloudinit.readthedocs.io/en/latest/topics/examples.html#run-commands-on-first-boot):

"runcmd":

#cloud-config

# run commands
# default: none
# runcmd contains a list of either lists or a string
# each item will be executed in order at rc.local like level with
# output to the console
# - runcmd only runs during the first boot
# - if the item is a list, the items will be properly executed as if
#   passed to execve(3) (with the first arg as the command).
# - if the item is a string, it will be simply written to the file and
#   will be interpreted by 'sh'
#
# Note, that the list has to be proper yaml, so you have to quote
# any characters yaml would eat (':' can be problematic)
runcmd:
 - [ ls, -l, / ]
 - [ sh, -xc, "echo $(date) ': hello world!'" ]
 - [ sh, -c, echo "=========hello world'=========" ]
 - ls -l /root
 - [ wget, "http://slashdot.org", -O, /tmp/index.html ]

"bootcmd":

#cloud-config

# boot commands
# default: none
# this is very similar to runcmd, but commands run very early
# in the boot process, only slightly after a 'boothook' would run.
# bootcmd should really only be used for things that could not be
# done later in the boot process.  bootcmd is very much like
# boothook, but possibly with more friendly.
# - bootcmd will run on every boot
# - the INSTANCE_ID variable will be set to the current instance id.
# - you can use 'cloud-init-per' command to help only run once
bootcmd:
 - echo 192.168.1.130 us.archive.ubuntu.com >> /etc/hosts
 - [ cloud-init-per, once, mymkfs, mkfs, /dev/vdb ]

also note the "cloud-init-per" command example in bootcmd. From it's help:

Usage: cloud-init-per frequency name cmd [ arg1 [ arg2 [ ... ] ]
   run cmd with arguments provided.

   This utility can make it easier to use boothooks or bootcmd
   on a per "once" or "always" basis.

   If frequency is:
      * once: run only once (do not re-run for new instance-id)
      * instance: run only the first boot for a given instance-id
      * always: run every boot