How to prevent duplicate SQS Messages?

Question

What is the best way to prevent duplicate messages in Amazon SQS? I have a SQS of domains waiting to be crawled. before I add a new domain to the SQS I can check with the saved

Answer 1

Amazon SQS Introduces FIFO Queues with Exactly-Once Processing and Lower Prices for Standard Queues

Using the Amazon SQS Message Deduplication ID The message deduplication ID is the token used for deduplication of sent messages. If a message with a particular message deduplication ID is sent successfully, any messages sent with the same message deduplication ID are accepted successfully but aren't delivered during the 5-minute deduplication interval.

Amazon SQS Introduces FIFO Queues

Using the Amazon SQS Message Deduplication ID