This could be a massive list - read Joe Duffy's excellent "Concurrent Programming On Windows" for much more detail. This is pretty much a brain dump...
- Try to avoid calling into significant chunks of code while you own a lock
- Avoid locking on references which code outside the class might also lock on
- If you ever need to acquire more than one lock at a time, always acquire those locks in the same order
- Where reasonable, use immutable types - they can be shared freely between threads
- Other than immutable types, try to avoid the need to share data between threads
- Avoid trying to make your types threadsafe; most types don't need to be, and usually the code which needs to share data will need to control the locking itself
- In a WinForms app:
- Don't perform any long-running or blocking operations on the UI thread
- Don't touch the UI from any thread other than the UI thread. (Use BackgroundWorker, Control.Invoke/BeginInvoke)
- Avoid thread-local variables (aka thread-statics) where possible - they can lead to unexpected behaviour, particularly on ASP.NET where a request may be served by different threads (search for "thread agility" and ASP.NET)
- Don't try to be clever. Lock-free concurrent code is hugely difficult to get right.
- Document the threading model (and thread safety) of your types
- Monitor.Wait should almost always be used in conjunction with some sort of check, in a while loop (i.e. while (I can't proceed) Monitor.Wait(monitor))
- Consider the difference between Monitor.Pulse and Monitor.PulseAll carefully every time you use one of them.
- Inserting Thread.Sleep to make a problem go away is never a real fix.
- Have a look at "Parallel Extensions" and the "Coordination and Concurrency Runtime" as ways of making concurrency simpler. Parallel Extensions is going to be part of .NET 4.0.
In terms of debugging, I don't have very much advice. Using Thread.Sleep to boost the chances of seeing race conditions and deadlocks can work, but you've got to have quite a reasonable understanding of what's wrong before you know where to put it. Logging is very handy, but don't forget that the code goes into a sort of quantum state - observing it via logging is almost bound to change its behaviour!