How to parse mixed text and JSON log entries in AWS CloudWatch for Log Metric Filter

Question

I am trying to parse log entries which are mix of text and JSON. First line is text representation and next lines are JSON payload of the event. One of the possible examples are:

2016-07-24T21:08:07.888Z [INFO] Command completed lessonrecords-create
{
  "key": "lessonrecords-create",
  "correlationId": "c1c07081-3f67-4ab3-a5e2-1b3a16c87961",
  "result": {
    "id": "9457ce88-4e6f-4084-bbea-14fff78ce5b6",
    "status": "NA",
    "private": false,
    "note": "Test note",
    "time": "2016-02-01T01:24:00.000Z",
    "updatedAt": "2016-07-24T21:08:07.879Z",
    "createdAt": "2016-07-24T21:08:07.879Z",
    "authorId": null,
    "lessonId": null,
    "groupId": null
  }
}

For this records I try define Log Metric Filter to a) match records b) select data or dimensions if possible.

According to the AWS docs JSON pattern should look like this:

{ $.key = "lessonrecords-create" }

however it does not match anything. My guess is that because of mix text and JSON in single log entry.

So, the questions are: 1. Is it possible to define pattern that will match this log format? 2. Is it possible to extract dimensions, values from such log format? 3. Help me with pattern to do this.

Answer 1

If you set up the metric filter in the way that you have defined, the test will not register any matches (I have also had this issue), however when you deploy the metric filter it will still register matches (at least mine did). Just keep in mind that there is no way (as far as I am aware) to run this metric filter BACKWARDS (ie. it will only capture data from when it is created). [If you're trying to get stats on past data, you're better off using log insight queries]

I am currently experimenting with different parse statements to try and extract data (its also a mix of JSON and text), this thread MAY help you (it didn't for me) Amazon Cloudwatch Logs Insights with JSON fields .

UPDATE! I have found a way to parse the text but its a little bit clunky. If you export your cloudwatch logs using a lamda function to SumoLogic, their search tool allows for MUCH better log manipulation and lets you parse JSON fields (if you treat the entire entry as text). SumoLogic is also really helpful because you can just extract your search results as a CSV. For my purposes, I parse the entire log message in SumoLogic, extract all the logs as a CSV and then I used regex in Python to filter through and extract the values I need.