SNS to Lambda vs SNS to SQS to Lambda

Question

I\'m trying to understand whether I need SQS in my workflow if someone can help explain. In my app, when an action is taken, it submits info to SNS topic which invokes LAMBDA to

Answer 1

I think couple of things changed in 2019 and SQS can trigger lambda via event source mapping which is mentioned by @alexs. Related blog post: https://aws.amazon.com/about-aws/whats-new/2018/04/aws-lambda-now-supports-amazon-sqs-as-event-source/

To summarize, you can use SQS to lambda with following benefits:

• Reprocessing events in case of failure and configure how many times a message should be retried before you give up (receive count)
• Longer retention period
• Typically chosen in the scenarios where there is a long running job and the lambda polls one by one from job queue.

you can choose to use SNS:

• If you need to fanout a single message to multiple destinations, say X message should be processed by Y and Z applications. I feel this is the biggest advantage, and if you want reliability in this, you can couple SNS and SQS together.
• You do not care about lost messages. Remember that there are still retry stratergies when using SNS(linear, geometric, exponential etc)
• Typically used in the cases where you can ingest/process messages faster. This can sometimes be a problem as well; imagine a scenario where there is a SNS notification for every email that your business receives and you dont have enough lambda concurrency to process all of them. You can solve this by putting an SQS to consume at your own pace.

In both the cases, there can be duplicate messages(in the cases of retry) and there cannot be order guarantees. If you need one, consider Kinesis streams.