Ubuntu AMI not loading user-data

Question

I am attempting to load user data into an Ubuntu 12.04 LTS AMI (ami-a29943cb, but I\'ve tried a few others to no avail) via boto\'s ec2.run_instances(..., user_data=USER_DATA).

Answer 1

Quick checklist:

1 ) SSH into your instance.
2 ) Run cat on /var/log/cloud-init-output.log and dump it into a nice text file.
3 ) You'll see multiple block which contains the structure below:

Dependencies Resolved

================================================================================
 Package                    Arch      Version               Repository     Size
================================================================================
Installing:


Transaction Summary
================================================================================
Install  X Package (+Y Dependent packages)

Each block is created according to the user data which you provided to the EC2 instance (All scripts are stored under /var/lib/cloud/instance/scripts).

4 ) Search for keywords like Failed, Error, WARNING or /var/lib/cloud/instance/scripts/.

For example in case of the error below:

/var/lib/cloud/instance/scripts/part-001: line 10: vncpasswd: command not found
cp: cannot stat '/lib/systemd/system/vncserver@.service': No such file or directory
sed: can't read /etc/systemd/system/vncserver@.service: No such file or directory
Failed to execute operation: No such file or directory
Failed to start vncserver@:1.service: Unit not found.
Loaded plugins: extras_suggestions, langpacks, priorities, update-motd
Cleaning repos: amzn2-core amzn2extra-docker amzn2extra-epel

---

(*) You can also quickly review the end of /var/log/cloud-init-output.log or /var/log/cloud-init.log which contain general error messages which will inform you about the fact that an error has occurred while executing the user data scripts.

The end of /var/log/cloud-init-output.log in the error above:

Is this ok [y/d/N]: Exiting on user command
Your transaction was saved, rerun it with:
 yum load-transaction /tmp/yum_save_tx.2020-08-31.15-14.VpTvV1.yumtx
Aug 31 15:14:00 cloud-init[3532]: util.py[WARNING]: Failed running /var/lib/cloud/instance/scripts/part-001 [1]
Aug 31 15:14:00 cloud-init[3532]: cc_scripts_user.py[WARNING]: Failed to run module scripts-user (scripts in /var/lib/cloud/instance/scripts)
Aug 31 15:14:00 cloud-init[3532]: util.py[WARNING]: Running module scripts-user (<module 'cloudinit.config.cc_scripts_user' from '/usr/lib/python2.7/site-packages/cloudinit/config/cc_scripts_user.pyc'>) failed
Cloud-init v. 19.3-3.amzn2 finished at Mon, 31 Aug 2020 15:14:00 +0000. Datasource DataSourceEc2.  Up 87.44 seconds

And the end of /var/log/cloud-init.log:

Aug 31 15:14:00 cloud-init[3532]: util.py[DEBUG]: Failed running /var/lib/cloud/instance/scripts/part-001 [1]
    Traceback (most recent call last):
      File "/usr/lib/python2.7/site-packages/cloudinit/util.py", line 910, in runparts
        subp(prefix + [exe_path], capture=False, shell=True)
      File "/usr/lib/python2.7/site-packages/cloudinit/util.py", line 2105, in subp
        cmd=args)
    ProcessExecutionError: Unexpected error while running command.
    Command: ['/var/lib/cloud/instance/scripts/part-001']
    Exit code: 1
    Reason: -
    Stdout: -
    Stderr: -
    cc_scripts_user.py[WARNING]: Failed to run module scripts-user (scripts in /var/lib/cloud/instance/scripts)

Answer 2

It's hard to know what happened without an error message, but there are a few places you can look:

1. The file /var/log/cloud-init.log will usually contain any errors (e.g. boto import failure) that occurred during instance bootstrapping.
2. The directory /var/lib/cloud/instance will contain the raw scripts and user-data downloaded to the instance
3. You can View/Edit USER_DATA inside the AWS console by right-clicking the instance, to see if boto populated it correctly.

Looking in those places should help provide clarity.

I know Ubuntu 12.04 comes with boto 2.2.2:

root@vanilla-562c:/var/lib/cloud/instance# python
Python 2.7.3 (default, Apr 20 2012, 22:44:07) 
[GCC 4.6.3] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>> import boto
>>> boto.__version__
'2.2.2'

..but I wonder if it's actually accessible in your PYTHONPATH at runtime.