How to push the data from dynamodb through stream

Question

Below is the json file

[
    {
        "year": 2013,
        "title": "Rush",
        "actors": [
                "Da         


        

Answer 1

This lambda just writing the document to DynamoDb, and I will not recommend adding the code in this lambda to push the same object to Elastic search, as lambda function should perform a single task and pushing the same document to ELK should be managed as a DynamoDB stream.

• What if ELK is down or not available how you will manage this in lambda?
• What if you want to disable this in future? you will need to modify lambda instead of controlling this from AWS API or AWS console, all you need to just disable the stream when required no changes on above lambda side code
• What if you want to move only modify or TTL item to elastic search?

So create Dyanodb Stream that pushes the document to another Lambda that is responsible to push the document to ELK, with this option you can also push old and new both items.

You can look into this article too that describe another approach data-streaming-from-dynamodb-to-elasticsearch

For above approach look into this GitHub project dynamodb-stream-elasticsearch.

const { pushStream } = require('dynamodb-stream-elasticsearch');

const { ES_ENDPOINT, INDEX, TYPE } = process.env;

function myHandler(event, context, callback) {
  console.log('Received event:', JSON.stringify(event, null, 2));
  pushStream({ event, endpoint: ES_ENDPOINT, index: INDEX, type: TYPE })
    .then(() => {
      callback(null, `Successfully processed ${event.Records.length} records.`);
    })
    .catch((e) => {
      callback(`Error ${e}`, null);
    });
}

exports.handler = myHandler;