Code is behaving differently in Release vs Debug Mode

Question

We have some unit tests that fail when run in Release mode vs debug mode. If I attach a debugger in release mode the tests pass. There is way too much code to publish here s

Answer 1

One thing that might cause the behaviour that you are seeing is an error that causes a race condition. Attaching a debugger can change the timing of the code such that the race condition is no longer triggered.

To fix it, use synchronization appropriately whenever you have multiple threads accessing data.

---

I am comparing some float values in the IsEqual method.

That sounds like a very bad idea. You should not compare floats for equality because floating point calcualtions are not 100% precise and you can get representation and rounding errors. Compare to see whether they are sufficiently close together. For calculations involving money, you probably want to use the decimal type instead.

Answer 2

Questions you should ask yourself -

1. Is my code threaded? Timing differences will affect output
2. Is someone calling Debug.Assert() with an expression that side effects?
3. What objects implement IDisposable() and do some do so in such a way that changes state?
4. Are you P/Invoking into unmanaged code?

Number 3 is a very likely bad-boy in this case. Garbage collection may be very different in debug and release and you may find that when an object is garbage collected is affecting the outcome of a later unit test.

And FYI, if you're using NUnit and TestDriven.NET - the two run tests in different orders.

Answer 3

This is often the case as the debug build is not optimized by default, and even if you enable it, the behavior when debugging is very different. You can disable the "Optimize code" from the project settings for all assemblies on the Properties->Build tab.

There are certainly other changes that can cause differences, like you mention Conditional Methods are one. These I've found to rarely be the cause of issues, for me it's almost always the optimizer.

Classic gotcha's of the optimizer include methods that get 'inlined' so that they fail to appear on a call stack. This causes problems when using System.Diagnostics.StackFrame classes to determine the current execution point. Similarly this will affect the result of MethodBase.GetCurrentMethod or other functions/behavior that rely on the executing method.

Then there are of course many things I've seen the optimizer do that I simply cannot explain at all. One such example was documented and discussed in a post 'HashDerivedBytes - replacing Rfc2898DeriveBytes, but why?' but I've never solved the mystery. I only know that the optimizer just flat broke Rfc2898DeriveBytes when used to generate a series of derived bytes. Oddly enough this only broke when the bytes generated were not evenly divisible by the size of the hash algorithm used (20) and only produced incorrect results after the first 20 bytes.

The fact is that optimizations adversely affecting code is not a new thing for compilers. Most of the old-school C++ developers will tell you that straight away and then, as I did, go into some long drawn out story about how they worked around it ;)

Answer 4

As Mark suggests, this is usually a result of a timing-related issue, often a race condition or synchronization problem.

One common way to handle this sort of problem is to use "print" statements in the affected areas to show you what's going on. If the print statements (Console.WriteLine, Response.Write, logging, or whatever) make the problem go away, store the values in global variables and print the globals once the problem has shown up.

The most recent time this has happened to me was in code that was reading from a serial port. The debugging activity caused just enough of a change in timing to affect how bytes from the serial port were buffered, which changed how the buffer was getting parsed. Since the print statements changed the timing, I had to store the data up to output later.

Answer 5

Just to add my two cents to this, I recently found that I had a date comparison in an sql procedure that the testing called. The dates were all auto-generated prior in the test procedure and values were inserted into the DB, and so occasionally they were exactly the same (when using RunTests) causing a null to be returned on a table join. Not what I was expecting. Obviously, in debug mode, since I'm slowly progressing through it, there will be a difference in the auto-generated times which meant that I never bumped into the error. I resolved this by inserting

Threading.Thread.Sleep(520)

wherever there would definitely be a delay between actions. Problem fixed.

Answer 6

Since it seems to be floating point related there are so many things that can go wrong. See: C# - Inconsistent math operation result on 32-bit and 64-bit and Double precision problems on .NET

There are so many things that can be trashed with floating points. And comparing floats for equality is a general no-no. You chould check the difference smaller than a reasonably epsilon.