amazon-sqs

AWS lambda read parameter or outputs from CloudFormation

故事扮演 提交于 2021-02-19 04:53:45
问题 Looks as really simple task but it's difficult to find good example on it. So, the task is following: AWS lambda puts some message to AWS-SQS. Code of AWS lambda contains such line: var QUEUE_URL = 'https://sqs.us-west-2.amazonaws.com/ID/QUEUE_NAME';", In order to get rid of this code there are possible two options: Create query that will lookup this queue based on region and queue name SQS has predictable names; Create Cloud Formaion script and specify these dependencies there. Based on this

Trigger Lambda Function in AWS when the message is present in SQS Queue

落爺英雄遲暮 提交于 2021-02-18 06:59:13
问题 I am using AWS Lambda function to process the messages in Queue it's working fine. But i need to execute this Lambda function when messages available or added in SQS queue. Is it possible to trigger the Lambda function based on SQS queue.Please suggest one method to achieve this goal. 回答1: Invoking Lambda functions from SQS queues is not directly supported. You can see the list of available triggers here: http://docs.aws.amazon.com/lambda/latest/dg/invoking-lambda-function.html Possible

celery monitoring with sqs broker

一个人想着一个人 提交于 2021-02-10 20:32:41
问题 We are using Airflow(1.10.3) with celery executor(4.1.1 (latentcall)) and broker SQS. While debugging an issue we tried our hands on celery CLI and found out that SQS broker is not supported for any of the inspect commands or monitoring tool eg. Flower. Is there any way we can monitor the tasks or events on celery workers? We have tried curses monitor as below: celery events -b sqs:// But it shows no worker discovered,no tasks selected. Inspect Commands directly shows: Availability: RabbitMQ

Adding SQS Permissions with conditions using AWS CLI Command

﹥>﹥吖頭↗ 提交于 2021-02-10 18:43:03
问题 How can I add the below listed SQS permission using AWS CLI command? "Statement": [ { "Sid": "Sid8390000202", "Effect": "Allow", "Principal": "*", "Action": "SQS:*", "Resource": "arn:aws:sqs:us-east-1:12345678:example-queue", "Condition": { "ArnEquals": { "aws:SourceArn": "arn:aws:sns:us-east-1:73628827939:MySNS" } } } ] 回答1: You can save the file locally as set-queue-attributes.json with the following policy. { "Id": "Policy1564523767951", "Version": "2012-10-17", "Statement": [ { "Sid":

C# AWS SQS Client inaccessible due to it's protection level

自作多情 提交于 2021-02-08 19:52:07
问题 So I've been having an issue getting the example from AWS to work for C#. I copy the code below straight from their site and put it in my program, but when it calls client.ListQueues(request); it tells me that it's inaccessible due to it's protection level. I've spent time searching, but I can't figure out why this isn't work. Any help would be appreciated. var client = new AmazonSQSClient(); // List all queues that start with "aws". var request = new ListQueuesRequest { QueueNamePrefix =

Simulating message persistence in SNS using SQS

こ雲淡風輕ζ 提交于 2021-02-08 11:28:26
问题 We are evaluating SNS for our messaging requirements to integrate multiple applications. we have a single producer that publishes messages to multiple topics on SNS. Each topic has 2-5 subscribers. In event of subscriber failures (down for maintenance) I have a few questions on the recommended strategy of using SQS queues per consumer Is it possible to configure SNS to push to SQS only in event of failure in delivering the message to a subscriber? Dumping all the messages in SQS queue creates

Simulating message persistence in SNS using SQS

北城余情 提交于 2021-02-08 11:28:26
问题 We are evaluating SNS for our messaging requirements to integrate multiple applications. we have a single producer that publishes messages to multiple topics on SNS. Each topic has 2-5 subscribers. In event of subscriber failures (down for maintenance) I have a few questions on the recommended strategy of using SQS queues per consumer Is it possible to configure SNS to push to SQS only in event of failure in delivering the message to a subscriber? Dumping all the messages in SQS queue creates

How to prevent duplicate SQS Messages?

时间秒杀一切 提交于 2021-02-05 12:42:45
问题 What is the best way to prevent duplicate messages in Amazon SQS? I have a SQS of domains waiting to be crawled. before I add a new domain to the SQS I can check with the saved data to see if it has been crawled recently, to prevent duplicates. The problem is with the domains that have not been crawled yet. For example if there is 1000 domains in the queue that have not been crawled. Any of those links could be added again, and again and again. Which swells my SQS to hundreds of thousands of

AWS SQS Boto3 sending messages to dead letter manually

99封情书 提交于 2021-02-05 04:56:52
问题 So I am building a small application that uses SQS. I have a simple handler process that determines if a given message is considered processed, marked for retry (to be re-queued) or is not able to be processed (should be sent to dead letter). However based on the docs it would appear the only way to truly send a message to DL is by using a redrive policy which operates over # of receives a message has racked up. Because of the nature of my application, I could have several valid retries if my

QqsClient.SendMessageAsync without await doesn't work

房东的猫 提交于 2021-01-29 09:55:06
问题 I have a Lambda function and that lambda function is supposed to send messages to multiple queues in SQS and then exit, If I add await, then messages are sent to all queues, var sqsClient = ServerlessHelper.GetAmazonSqsClient(); foreach (var item in items) { await sqsClient.SendMessageAsync(item.QueueUrl, item.Message); } But if I remove await from the code, then none of messages are sent to queues. I want to send messages in parallel. But due to await I have function cannot send messages in