Inconsistency rounding real numbers in C #

Question

I have this test code:

class Test
{
    static void Main()
    {
        decimal m = 1M / 6M;
        double d = 1.0 / 6.0;

        decimal notQuiteWholeM =         


        

Answer 1

Not quite reading your question in the same way as the other two answers. The gist of it: Does the formatted string representation of a double "round" in C#?

Yes.

Internally double is represented with full IEEE-754 decimal digit precision (15-17 digits), which is why:

notQuiteWholeD < 1.0 == true    // because notQuiteWholeD = 0.99999999999999989

However, when formatting it as a string, by default it will use 15 digit precision - equivalent to:

String.Format("{0:G15}", notQuiteWholeD)   // outputs "1"

To get all the digits of the full internal representation, you can use:

Console.WriteLine("{0:G17}", notQuiteWholeD);

Or:

Console.WriteLine("{0:R}", notQuiteWholeD);

Both, in this case, will output "0,99999999999999989".

The former will always use 17 digit precision. The latter ("roundtrip precision") will use 15 digits if that's enough precision for the following to be true, otherwise it will use 17:

Double.Parse(String.Format("{0:G15}", notQuiteWholeD)) == notQuiteWholeD

Bonus Example: ... of when G17 and R differ:

Console.WriteLine("{0:G17}", 1.0000000000000699); // outputs "1.0000000000000699"
Console.WriteLine("{0:R}",   1.0000000000000699); // outputs "1.00000000000007"

1.0000000000000699 (17 significant digits) can be represented accurately enough for a roundtrip using only 15 significant digits. In other words, the double representation of 1.00...07 is the same as for 1.00...0699.

So 1.00...07 (15 digits) is a shorter input to get the exact same internal (17 digit) representation. That means R will round it to 15 digits, while G17 will keep all the digits of the internal representation.

Maybe it's clearer when realizing that this:

Console.WriteLine("{0:G17}", 1.00000000000007); // outputs "1.0000000000000699"
Console.WriteLine("{0:R}",   1.00000000000007); // outputs "1.00000000000007"

... gives the exact same results.

Answer 2

As we know, 1/6 = 0.1666 (repeating), decimal and double can not represent repeating numbers, they are calculated when assigned to. Since they are built from different backing data structures they represent a different set of possible numbers and round differently in some cases.

For this code:

Console.WriteLine(notQuiteWholeD < 1.0);   // Prints: True. Why?

Since notQuiteWholeD is 0.99999999999999989 it prints true.

I'm not going to cover how the double and decimal work behind the scenes but here is some reading material if you're interested.

• Double-precision floating-point format
• SO - Behind the scenes, what's happening with decimal value type in C#/.NET?
• Decimal floating point in .NET

Answer 3

Decimal is stored in terms of base 10. Double is stored in terms of base 2. Neither of those bases can exactly represent 1 / 6 with a finite representation.

That explains all the output except Console.WriteLine(notQuiteWholeD). You get "1" for output, even though the actual value stored is less than 1. Since the output is in base 10, it has to convert from base 2. Part of the conversion includes rounding.