How to implement a priority queue using SQS(Amazon simple queue service)

给你一囗甜甜゛ 提交于 2020-05-14 17:18:38

问题


I have a situation when a msg fails and I would like to replay that msg with the highest priority using python boto package so he will be taken first. If I'm not wrong SQS queue does not support priority queue, so I would like to implement something simple.

Important note: when a msg fails I no longer have the message object, I only persist the receipt_handle so I can delete the message(if there was more than x retries) / change timeout visibility in order to push him back to queue.

Thanks!


回答1:


I don't think there is any way to do this with a single SQS queue. You have no control over delivery of messages and, therefore, no way to impose a priority on messages. If you find a way, I would love to hear about it.

I think you could possibly use two queues (or more generally N queues where N is the number of levels of priority) but even this seems impossible if you don't actually have the message object at the time you determine that it has failed. You would need the message object so that the data could be written to the high-priority queue.

I'm not sure this actually qualifies as an answer 8^)




回答2:


As far as I know AWS SQS does not provide a native way or doing a priority queue (single queue priority). If you are open to considering other options, RabbitMQ can do this. Client can specify a priority of 0-255 in the message, and the queue will prioritize a higher priority message gets to the customer first.

For more information, please look at https://www.rabbitmq.com/priority.html




回答3:


By "when a msg fails", if you meant "processing failure" then you could look into Dead Letter Queue (DLQ) feature that comes with SQS. You can set the receive count threshold to move the failed messages to DLQ. Each DLQ is associated with an SQS.

In your case, you could make "max receive count" = 1 and you deal with that message seperately



来源:https://stackoverflow.com/questions/28970289/how-to-implement-a-priority-queue-using-sqsamazon-simple-queue-service

易学教程内所有资源均来自网络或用户发布的内容,如有违反法律规定的内容欢迎反馈
该文章没有解决你所遇到的问题?点击提问,说说你的问题,让更多的人一起探讨吧!