Dead-lettering dead-lettered messages in RabbitMQ

Question

Here's what we have here:

• Topic Exchange DLE, which is intended to be a Dead-Letter Exchange
• Topic Exchange E, which is the "main" Exchange
• Several Queues (EQ1, ..., EQn) bound to E (and initialized with x-dead-letter-exchange = DLE), each with own Routing Key. These queues are the ones being consumed from.
• For each EQn, there's a DLEQn (initialized with x-dead-letter-exchange = E and x-message-ttl = 5000), bound to DLE with the same routing key as EQn. These queues are not being consumed from

What I want is the following: if a consumer cannot process a message from EQn, it Nacks the message with requeue: false and it gets to the DLEQn - that is, to an appropriate queue on the Dead-Letter Exchange. Now, I want this message to sit on the DLEQn for some time and then get routed back to the original queue EQn to be processed again.

Try as I might, I could not get the "redelivery to the original queue" working. I see that messages sit in the DLEQn with all the right headers and Routing Key intact, but after TTL expires they just vanish into thin air.

What am I doing wrong here?

Answer 1

Yes, you can do this. We are currently doing this in production and it works great. The code is too long to include here but I will show you the diagram I created that represents the process. The basic idea is that the First DLX has a TTL, once that TTL expires the message goes into a 2nd queue to be re-sent back into the original.

Answer 2

RabbitMQ detects message flow cycling (E -> DLE -> E -> DLE ...) and silently drops messages:

From DLX manual (Routing Dead-Lettered Messages section):

It is possible to form a cycle of dead-letter queues. For instance, this can happen when a queue dead-letters messages to the default exchange without specifiying a dead-letter routing key. Messages in such cycles (i.e. messages that reach the same queue twice) will be dropped if the entire cycle is due to message expiry.