Inconsistency rounding real numbers in C #

前端 未结 3 1675
梦谈多话
梦谈多话 2021-01-14 03:38

I have this test code:

class Test
{
    static void Main()
    {
        decimal m = 1M / 6M;
        double d = 1.0 / 6.0;

        decimal notQuiteWholeM =         


        
3条回答
  •  时光说笑
    2021-01-14 04:33

    Decimal is stored in terms of base 10. Double is stored in terms of base 2. Neither of those bases can exactly represent 1 / 6 with a finite representation.

    That explains all the output except Console.WriteLine(notQuiteWholeD). You get "1" for output, even though the actual value stored is less than 1. Since the output is in base 10, it has to convert from base 2. Part of the conversion includes rounding.

提交回复
热议问题