Sql Server Service Broker: How to structure Conversations for a simple queue scenario?

Question

I\'m a Sql Server Service Broker novice and I\'m trying to grasp the best way to set Service Broker up for a (seemingly) simple use case: I want to create a simple work queue, w

Answer 1

I really like Remus' answer, though it doesn't especially touch on why you might prefer starting a separate conversation per work item, rather than putting all work items in a single conversation. Two notes related to this:

First, putting all work items into a single conversation will probably cause concurrency problems if you have multiple threads/processes processing work items. Service broker worker processes tend to look like this (in pseudocode):

begin transaction
receive top n work items from queue
process work items
commit transaction

(By not committing until work items are successfully processed, you ensure, for example, that if your process dies then the work items it has received but not yet processed won't get dropped from the queue.)

The concurrency problem would arise because service broker is programmed such that each RECEIVE command acquires an exclusive read lock on all messages in the queue that share the same conversation (or conversation group) as the ones that were RECEIVEd. That lock is held until the transaction is committed. (See Conversation Group Locks.) So if all work items in the queue are in a single conversation, then while one worker process is in the "process work items" step, no other worker processes can be doing any work.

A second issue with putting lots of items into a single conversation is that it increases the amount of work items you might lose or have to reprocess in certain error conditions. To describe this properly I defer to Remus; see his Recycling Conversations, especially the part that says "reusing a single dialog to send all your messages [...] is like putting all your eggs in one basket." You might be able to recover from some of these error situations, but it will probably introduce more complexity to your code.

There are probably a few more arguments to be made against using a single conversation for all work items, but I'm not as familiar with with them.

This is not to say the correct solution is always to start a separate conversation for each and every work item. After having read through Remus' posts, though, his advice seems sound; start with one work item per conversation, and then add complexity if required. (But probably in no case should you go to the extreme of putting all messages in a single conversation.)