In a FIFO Qeueing system, what's the best way the to implement priority messaging

Question

For message-oriented middleware that does not consistently support priority messages (such as AMQP) what is the best way to implement priority consumption when queues have o

Answer 1

Given only FIFO support for a given single queue, you will of course have to introduce either multiple queues, an intermediary, or have a more complex consumer.

Multiple queues could be handled in a couple of ways. The producer and consumer could agree to have two queues between them, one for high-priority, and one for background tasks.

If your producer is constrained to a single queue, but you have control over the consumer, consider introducing a fan-out router in the path. So producer->Router is a single queue, and the router then has two queues to the consumer.

Another way to tackle it, which is likely less than ideal, would be to have your consumer spin a thread to front the queue, then dispatch the work internally. Something like the router version above, but inside a single app. This has the downside of having multiple messages in flight inside your app, which may complicate recovery in the event of a failure.

Don't forget to consider starvation of the effectively low-priority events, whatever they are, if some of them should be processed even if there are higher-priority events still hanging about.