How should I unit test threaded code?

Question

I have thus far avoided the nightmare that is testing multi-threaded code since it just seems like too much of a minefield. I\'d like to ask how people have gone about test

Answer 1

Assuming under "multi-threaded" code was meant something that is

• stateful and mutable
• AND accessed/modified by multiple threads concurrently

In other words we are talking about testing custom stateful thread-safe class/method/unit - which should be a very rare beast nowadays.

Because this beast is rare, first of all we need to make sure that there are all valid excuses to write it.

Step 1. Consider modifying state in same synchronization context.

Today it is easy to write compose-able concurrent and asynchronous code where IO or other slow operations offloaded to background but shared state is updated and queried in one synchronization context. e.g. async/await tasks and Rx in .NET etc. - they are all testable by design, "real" Tasks and schedulers can be substituted to make testing deterministic (however this is out of scope of the question).

It may sound very constrained but this approach works surprisingly well. It is possible to write whole apps in this style without need to make any state thread-safe (I do).

Step 2. If manipulating of shared state on single synchronization context is absolutely not possible.

Make sure the wheel is not being reinvented / there's definitely no standard alternative that can be adapted for the job. It should be likely that code is very cohesive and contained within one unit e.g. with a good chance it is a special case of some standard thread-safe data structure like hash map or collection or whatever.

Note: if code is large / spans across multiple classes AND needs multi-thread state manipulation then there's a very high chance that design is not good, reconsider Step 1

Step 3. If this step is reached then we need to test our own custom stateful thread-safe class/method/unit.

I'll be dead honest : I never had to write proper tests for such code. Most of the time I get away at Step 1, sometimes at Step 2. Last time I had to write custom thread-safe code was so many years ago that it was before I adopted unit testing / probably I wouldn't have to write it with the current knowledge anyway.

If I really had to test such code (finally, actual answer) then I would try couple of things below

1. Non-deterministic stress testing. e.g. run 100 threads simultaneously and check that end result is consistent. This is more typical for higher level / integration testing of multiple users scenarios but also can be used at the unit level.

2. Expose some test 'hooks' where test can inject some code to help make deterministic scenarios where one thread must perform operation before the other. As ugly as it is, I can't think of anything better.

3. Delay-driven testing to make threads run and perform operations in particular order. Strictly speaking such tests are non-deterministic too (there's a chance of system freeze / stop-the-world GC collection which can distort otherwise orchestrated delays), also it is ugly but allows to avoid hooks.