Any satisfactory approaches to unit testing thread safety in Java?

Question

I am looking at improving a package that I believe not to be threadsafe when its input is shared between multiple worker threads. According to TDD principles, I should write so

Answer 1

There are two basic problems you have to solve. The first is this: how do you generate a thread clash? Second, how do you validate your test?

The first one is straightforward. Use a big hammer. Write some code that's capable of detecting the case where one thread steps on another, and run that code 50 times or so on 50 consecutive threads. You can do this with a countdown latch:

public void testForThreadClash() {
  final CountDownLatch latch = new CountDownLatch(1);
  for (int i=0; i<50; ++i) {
    Runnable runner = new Runnable() {
      public void run() {
        try {
          latch.await();
          testMethod();
        } catch (InterruptedException ie) { }
      }
    }
    new Thread(runner, "TestThread"+i).start();
  }
  // all threads are waiting on the latch.
  latch.countDown(); // release the latch
  // all threads are now running concurrently.
}

Your testMethod() has to generate a situation where, without the synchronized block, some thread will step on another thread's data. It should be able to detect this case and throw an exception. (The code above doesn't do this. The exception will most likely be generated on one of the test threads, and you need to put in a mechanism that will detect it on the main thread, but that's a separate issue. I left it out for simplicity, but you can do this with a second latch, which the test can prematurely clear on an exception.)

The second question is trickier, but there's a simple solution. The question is this: How do you know that your hammer will generate a clash? Of course, your code has synchronized blocks to prevent a clash, so you can't answer the question. But you can. Just remove the synchronized keywords. Since it's the synchronized keyword that makes your class threadsafe, you can just remove them and re-run the test. If the test is valid, it will now fail.

When I first wrote the code above, it turned out to be invalid. I never saw a clash. So it's not (yet) a valid test. But now we know how to get a clear answer to the second question. We're getting the wrong answer, but we can now tinker with the test to generate the failure we're looking for. Here's what I did: I just ran the test 100 times in a row.

for (int j=0; j<100; ++j) {
  testForThreadClash();
}

Now my test failed reliably on about the 20th iteration or so. This confirms my test is valid. I can now restore the synchronized keyword and rerun the test, confident that it will tell me if my class is threadsafe.