Unit-testing multithreaded applications

Question

Has anyone got any advice or know of any frameworks for unit-testing of multithreaded applications?

Answer 1

Typically you don't unit test the concurrency of multi threaded applications as the unit tests aren't dependable and reproducible - because of the nature of concurrency bugs its not generally possible to write unit tests that consistently either fail or succeed, and so unit tests of concurrent code generally don't make very useful unit tests.

Instead you unit test each single threaded components of your application as normal and rely on load testing sessions to identify concurrency issues.

That said, there are some experimental load testing frameworks for testing concurrent applications, such as Microsoft CHESS - CHESS repeatedly runs a given unit test and systematically explores every possible interleaving of a concurrent test. This makes your unit tests dependable and reproducible.

For the moment however CHESS is still experimental (and probably not usable with the JVM) - for now stick with load testing to weed out concurrency issues.

Answer 2

Try multithreadedTC

http://code.google.com/p/multithreadedtc/

http://code.google.com/p/multithreadedtc-junit4/

Answer 3

Do not test multithreaded applications. Refactor the code to remove coupling between work that is done in different threads. Then test it separately.

Answer 4

There's nothing inherently wrong with testing multi-threaded code, especially if threading is the point of the code you're testing. The general approach for testing threaded/async code is to block the main test thread, capture any failed assertions from other threads, unblock the main test thread and rethrow any failures. ConcurrentUnit takes care of most of this for you:

  final Waiter waiter = new Waiter();

  new Thread(() -> {
    doSomeWork();
    waiter.assertTrue(true);
    waiter.resume();
  }).start();

  // Wait for resume() to be called
  waiter.await(1000);