List out auto scaling group names with a specific application tag using boto3

Question

I was trying to fetch auto scaling groups with Application tag value as 'CCC'.

The list is as below,

gweb
prd-dcc-eap-w2
gweb
prd-dcc-emc
gweb
prd-dcc-ems
CCC
dev-ccc-wer
CCC
dev-ccc-gbg
CCC
dev-ccc-wer

The script I coded below gives output which includes one ASG without CCC tag.

#!/usr/bin/python
import boto3

client = boto3.client('autoscaling',region_name='us-west-2')

response = client.describe_auto_scaling_groups()

ccc_asg = []

all_asg = response['AutoScalingGroups']
for i in range(len(all_asg)):
    all_tags = all_asg[i]['Tags']
    for j in range(len(all_tags)):
        if all_tags[j]['Key'] == 'Name':
                asg_name = all_tags[j]['Value']
        #        print asg_name
        if all_tags[j]['Key'] == 'Application':
                app = all_tags[j]['Value']
        #        print app
        if all_tags[j]['Value'] == 'CCC':
                ccc_asg.append(asg_name)

print ccc_asg

The output which I am getting is as below,

['prd-dcc-ein-w2', 'dev-ccc-hap', 'dev-ccc-wfd', 'dev-ccc-sdf']

Where as 'prd-dcc-ein-w2' is an asg with a different tag 'gweb'. And the last one (dev-ccc-msp-agt-asg) in the CCC ASG list is missing. I need output as below,

dev-ccc-hap-sdf
dev-ccc-hap-gfh
dev-ccc-hap-tyu
dev-ccc-mso-hjk

Am I missing something ?.

Answer 1

In boto3 you can use Paginators with JMESPath filtering to do this very effectively and in more concise way.

From boto3 docs:

JMESPath is a query language for JSON that can be used directly on paginated results. You can filter results client-side using JMESPath expressions that are applied to each page of results through the search method of a PageIterator.

When filtering with JMESPath expressions, each page of results that is yielded by the paginator is mapped through the JMESPath expression. If a JMESPath expression returns a single value that is not an array, that value is yielded directly. If the result of applying the JMESPath expression to a page of results is a list, then each value of the list is yielded individually (essentially implementing a flat map).

Here is how it looks like in Python code with mentioned CCP value for Application tag of Auto Scaling Group:

import boto3

client = boto3.client('autoscaling')
paginator = client.get_paginator('describe_auto_scaling_groups')
page_iterator = paginator.paginate(
    PaginationConfig={'PageSize': 100}
)

filtered_asgs = page_iterator.search(
    'AutoScalingGroups[] | [?contains(Tags[?Key==`{}`].Value, `{}`)]'.format(
        'Application', 'CCP')
)

for asg in filtered_asgs:
    print asg['AutoScalingGroupName']

Answer 2

Elaborating on Michal Gasek's answer, here's an option that filters ASGs based on a dict of tag:value pairs.

def get_asg_name_from_tags(tags):
    asg_name = None
    client = boto3.client('autoscaling')
    while True:

        paginator = client.get_paginator('describe_auto_scaling_groups')
        page_iterator = paginator.paginate(
            PaginationConfig={'PageSize': 100}
        )
        filter = 'AutoScalingGroups[]'
        for tag in tags:
            filter = ('{} | [?contains(Tags[?Key==`{}`].Value, `{}`)]'.format(filter, tag, tags[tag]))
        filtered_asgs = page_iterator.search(filter)
        asg = filtered_asgs.next()
        asg_name = asg['AutoScalingGroupName']
        try:
            asgX = filtered_asgs.next()
            asgX_name = asg['AutoScalingGroupName']
            raise AssertionError('multiple ASG\'s found for {} = {},{}'
                     .format(tags, asg_name, asgX_name))
        except StopIteration:
            break
    return asg_name

eg:

asg_name = get_asg_name_from_tags({'Env':env, 'Application':'app'})

It expects there to be only one result and checks this by trying to use next() to get another. The StopIteration is the "good" case, which then breaks out of the paginator loop.

Answer 3

I got it working with below script.

#!/usr/bin/python
import boto3

client = boto3.client('autoscaling',region_name='us-west-2')

response = client.describe_auto_scaling_groups()

ccp_asg = []

all_asg = response['AutoScalingGroups']
for i in range(len(all_asg)):
    all_tags = all_asg[i]['Tags']
    app = False
    asg_name = ''
    for j in range(len(all_tags)):
        if 'Application' in all_tags[j]['Key'] and all_tags[j]['Value'] in ('CCP'):
                app = True
        if app:
                if 'Name' in all_tags[j]['Key']:
                        asg_name = all_tags[j]['Value']
                        ccp_asg.append(asg_name)
print ccp_asg

Feel free to ask if you have any doubts.