Ways to Find a Race Condition

Question

I have a bit of code with a race condition in it... I know that it is a race condition because it does not happen consistently, and it seems to happen more often on dual core ma

Answer 1

Put sleeps in various parts of your code. Something that is threadsafe will be threadsafe even if it (or asynchronous code) sleeps for even seconds.

Answer 2

I've had some luck with using Visual Studio's tracepoints to find race conditions. Of course it still affects the timing, but in the cases I used it, at least, it wasn't enough to completely prevent the race conditions from occurring. It seemed less disruptive than dedicated logging, at least.

Other than that, try posting the code allowing others to look over it. Just studying the code in detail isn't a bad way to find race conditions.

Answer 3

The best way I know of to track these down is to use CHESS in Visual Studio. This is not a simple tool to use, and will probably require testing subsections of your app progressively. Good luck.

Answer 4

You can use tools like Intel Inspector which are able to check for certain types of race conditions.

Answer 5

So, the sledgehammer method for me has been the following, which takes a lot of patience and can in the best case scenario get you on the right track. I used this to figure out what was going on with this particular problem. I have been using tracepoints, one at the beginning of the suspected high-level function, and one at the end. Move the tracepoint down. If adding the tracepoint at the beginning of the function causes your bug to stop happening, move the tracepoint down until you can reproduce the condition again. The idea is that the tracepoint will not affect timing if you place it after the call that eventually triggers unsafe code, but will if you place it before. Also, note your output window. Between what messages is your bug occuring? You can use tracepoints to narrow this range as well.

Once you narrow your bug down to a manageable region of code, you can throw in breakpoints and have a look at what the other threads are up to at this point.

Answer 6

There is a tool included in CLang and gcc 4.8+ called ThreadSanitizer.

You compile your code using the -fsanitize=thread flag

Example:

$ cat simple_race.cc
#include <pthread.h>
#include <stdio.h>

int Global;

void *Thread1(void *x) {
  Global++;
  return NULL;
}

void *Thread2(void *x) {
  Global--;
  return NULL;
}

int main() {
  pthread_t t[2];
  pthread_create(&t[0], NULL, Thread1, NULL);
  pthread_create(&t[1], NULL, Thread2, NULL);
  pthread_join(t[0], NULL);
  pthread_join(t[1], NULL);
}

And the output

$ clang++ simple_race.cc -fsanitize=thread -fPIE -pie -g
$ ./a.out 
==================
WARNING: ThreadSanitizer: data race (pid=26327)
  Write of size 4 at 0x7f89554701d0 by thread T1:
    #0 Thread1(void*) simple_race.cc:8 (exe+0x000000006e66)

  Previous write of size 4 at 0x7f89554701d0 by thread T2:
    #0 Thread2(void*) simple_race.cc:13 (exe+0x000000006ed6)

  Thread T1 (tid=26328, running) created at:
    #0 pthread_create tsan_interceptors.cc:683 (exe+0x00000001108b)
    #1 main simple_race.cc:19 (exe+0x000000006f39)

  Thread T2 (tid=26329, running) created at:
    #0 pthread_create tsan_interceptors.cc:683 (exe+0x00000001108b)
    #1 main simple_race.cc:20 (exe+0x000000006f63)
==================
ThreadSanitizer: reported 1 warnings