Threadsafe FIFO Queue/Buffer

Question

I need to implement a sort of task buffer. Basic requirements are:

• Process tasks in a single background thread
• Receive tasks from multiple threads

Answer 1

You could use Rx on .NET 3.5 for this. It might have never come out of RC, but I believe it is stable* and in use by many production systems. If you don't need Subject you might find primitives (like concurrent collections) for .NET 3.5 you can use that didn't ship with the .NET Framework until 4.0.

Alternative to Rx (Reactive Extensions) for .net 3.5

* - Nit picker's corner: Except for maybe advanced time windowing, which is out of scope, but buffers (by count and time), ordering, and schedulers are all stable.

Answer 2

I suggest you take a look at TPL DataFlow. BufferBlock is what you're looking for, but it offers so much more.

Answer 3

Look at my lightweight implementation of threadsafe FIFO queue, its a non-blocking synchronisation tool that uses threadpool - better than create own threads in most cases, and than using blocking sync tools as locks and mutexes. https://github.com/Gentlee/SerialQueue

Usage:

var queue = new SerialQueue();
var result = await queue.Enqueue(() => /* code to synchronize */);

Answer 4

You should think about ConcurrentQueue, which is FIFO, in fact. If not suitable, try some of its relatives in Thread-Safe Collections. By using these you can avoid some risks.

Answer 5

You can actually handle this with the out-of-the-box BlockingCollection.

It is designed to have 1 or more producers, and 1 or more consumers. In your case, you would have multiple producers and one consumer.

When you receive a stop signal, have that signal handler

• Signal producer threads to stop
• Call CompleteAdding on the BlockingCollection instance

The consumer thread will continue to run until all queued items are removed and processed, then it will encounter the condition that the BlockingCollection is complete. When the thread encounters that condition, it just exits.