How do you structure sequential AWS service calls within lambda given all the calls are asynchronous?

Question

I\'m coming from a java background so a bit of a newbie on Javascript conventions needed for Lambda.

I\'ve got a lambda function which is meant to do several AWS tasks i

Answer 1

By default Javascript is asynchronous.

So, everything that you have to do, it's not to use those libraries, you can, but there's simple ways to solve this. In this code, I sent the email, with the data that comes from the event, but if you want, you just need to add more functions inside functions.

What is important is the place where your context.done(); is going to be, he is going to end your Lambda function. You need to put him in the end of the last function.

var AWS = require('aws-sdk');    
AWS.config.credentials = { "accessKeyId": "AAAA","secretAccessKey": "BBBB"};
AWS.config.region = 'us-east-1';
var ses = new AWS.SES({apiVersion: '2010-12-01'});

exports.handler = function(event, context) {

    console.log(event.nome);
    console.log(event.email);
    console.log(event.mensagem);

    nome = event.nome;
    email = event.email;
    mensagem = event.mensagem;

    var to = ['email@company.com.br'];
    var from = 'site@company.com.br';

    // Send email
    mensagem = ""+nome+"||"+email+"||"+mensagem+"";

    console.log(mensagem);
    ses.sendEmail( { 
       Source: from, 
       Destination: { ToAddresses: to },
       Message: {
           Subject: {
              Data: 'Form contact our Site'
           },
           Body: {
               Text: {
                   Data: mensagem,
               }
            }
       }
    },
    function(err, data) {
        if (err) {
            console.log("ERROR="+err, err.stack); 
            context.done();
          } else {
            console.log("EMAIL SENT="+data);
            context.done();
          }
     });
}

Answer 2

I don't know Lambda but you should look into the node async library as a way to sequence asynchronous functions.

async has made my life a lot easier and my code much more orderly without the deep nesting issue you mentioned in your question.

Typical async code might look like:

async.waterfall([
    function doTheFirstThing(callback) {
         db.somecollection.find({}).toArray(callback);
    },
    function useresult(dbFindResult, callback) {
         do some other stuff  (could be synch or async)
         etc etc etc
         callback(null);
],
function (err) {
    //this last function runs anytime any callback has an error, or if no error
    // then when the last function in the array above invokes callback.
    if (err) { sendForTheCodeDoctor(); }
});

Have a look at the async doco at the link above. There are many useful functions for serial, parallel, waterfall, and many more. Async is actively maintained and seems very reliable.

good luck!

Answer 3

I found this article which seems to have the answer in native javascript.

Five patterns to help you tame asynchronis javascript.

Answer 4

A very specific solution that comes to mind is cascading Lambda calls. For example, you could write:

1. A Lambda function gets something from DynamoDB, then invokes…
2. …a Lambda function that calls SNS to get an endpoint, then invokes…
3. …a Lambda function that sends a message through SNS, then invokes…
4. …a Lambda function that writes to DynamoDB

All of those functions take the output from the previous function as input. This is of course very fine-grained, and you might decide to group certain calls. Doing it this way avoids callback hell in your JS code at least.

(As a side note, I'm not sure how well DynamoDB integrates with Lambda. AWS might emit change events for records that can then be processed through Lambda.)

Answer 5

I like the answer from @jonathanbaraldi but I think it would be better if you manage control flow with Promises. The Q library has some convenience functions like nbind which help convert node style callback API's like the aws-sdk into promises.

So in this example I'll send an email, and then as soon as the email response comes back I'll send a second email. This is essentially what was asked, calling multiple services in sequence. I'm using the then method of promises to manage that in a vertically readable way. Also using catch to handle errors. I think it reads much better just simply nesting callback functions.

var Q = require('q');
var AWS = require('aws-sdk');    
AWS.config.credentials = { "accessKeyId": "AAAA","secretAccessKey": "BBBB"};
AWS.config.region = 'us-east-1';

// Use a promised version of sendEmail
var ses = new AWS.SES({apiVersion: '2010-12-01'});
var sendEmail = Q.nbind(ses.sendEmail, ses);

exports.handler = function(event, context) {

    console.log(event.nome);
    console.log(event.email);
    console.log(event.mensagem);

    var nome = event.nome;
    var email = event.email;
    var mensagem = event.mensagem;

    var to = ['email@company.com.br'];
    var from = 'site@company.com.br';

    // Send email
    mensagem = ""+nome+"||"+email+"||"+mensagem+"";

    console.log(mensagem);

    var params = { 
        Source: from, 
        Destination: { ToAddresses: to },
        Message: {
        Subject: {
            Data: 'Form contact our Site'
        },
        Body: {
            Text: {
                Data: mensagem,
            }
        }
    };

    // Here is the white-meat of the program right here.
    sendEmail(params)
        .then(sendAnotherEmail)
        .then(success)
        .catch(logErrors);

    function sendAnotherEmail(data) {
        console.log("FIRST EMAIL SENT="+data);

        // send a second one.
        return sendEmail(params);
    }

    function logErrors(err) {
        console.log("ERROR="+err, err.stack);
        context.done();
    }

    function success(data) {
        console.log("SECOND EMAIL SENT="+data);
        context.done();
    }
}

Answer 6

I would like to offer the following solution, which simply creates a nested function structure.

// start with the last action
var next = function() { context.succeed(); };

// for every new function, pass it the old one
next = (function(param1, param2, next) {
    return function() { serviceCall(param1, param2, next); };
})("x", "y", next);

What this does is to copy all of the variables for the function call you want to make, then nests them inside the previous call. You'll want to schedule your events backwards. This is really just the same as making a pyramid of callbacks, but works when you don't know ahead of time the structure or quantity of function calls. You have to wrap the function in a closure so that the correct value is copied over.

In this way I am able to sequence AWS service calls such that they go 1-2-3 and end with closing the context. Presumably you could also structure it as a stack instead of this pseudo-recursion.