how to view aws log real time (like tail -f)

Question

I can view the log using the following command.

aws logs get-log-events --log-group-name groupName --log-stream-name streamName --limit 100

wha

Answer 1

Because CloudWatch logs can be delayed (i.e. not "realtime" by precise definition) you parse the previous events for the last timestamp and start the next iteration there. This script uses aws logs get-log-events for which you must specify a valid stream_name.

#!/bin/bash
    
group_name=''
stream_name=''
start_seconds_ago=300

start_time=$(( ( $(date -u +"%s") - $start_seconds_ago ) * 1000 ))
while [[ -n "$start_time" ]]; do
    loglines=$(aws logs get-log-events --log-group-name "$group_name" --log-stream-name "$stream_name" --start-time $start_time --output text)
    [ $? -ne 0 ] && break
      next_start_time=$( sed -nE 's/^EVENTS.([[:digit:]]+).+$/\1/ p' <<< "$loglines" | tail -n1 )
    [ -n "$next_start_time" ] && start_time=$(( $next_start_time + 1 ))
    echo "$loglines"
    sleep 15
done

Or if you want to tail an entire log group, this script uses aws logs filter-log-events without a stream name:

#!/bin/bash

group_name=''
start_seconds_ago=300
  
start_time=$(( ( $(date -u +"%s") - $start_seconds_ago ) * 1000 ))
while [[ -n "$start_time" ]]; do
    loglines=$(aws logs filter-log-events --log-group-name "$group_name" --interleaved --start-time $start_time --output text)
    [ $? -ne 0 ] && break
    next_start_time=$( sed -nE 's/^EVENTS.([^[:blank:]]+).([[:digit:]]+).+$/\2/ p' <<< "$loglines" | tail -n1 )
    [ -n "$next_start_time" ] && start_time=$(( $next_start_time + 1 ))
    echo "$loglines"
    sleep 15
done

I've also put up the scripts that I use as GitHub gists: https://gist.github.com/tekwiz/964a3a8d2d84ff4c8b5288d9a703fbce.

Warning: the above code & scripts are written for my macOS system which is customized (bastardized??) with Homebrew and GNU coreutils, so some command options may need to be tweaked for your system. Edits are welcome :)