decimal rounding is off for (276/304)*304

后端 未结 3 1589
走了就别回头了
走了就别回头了 2021-01-18 06:32

If you put the following code in your compiler the result is a bit bizar:

decimal x = (276/304)*304;
double y = (276/304)*304;

Console.WriteLine("decima         


        
3条回答
  •  慢半拍i
    慢半拍i (楼主)
    2021-01-18 06:58

    276/304 = 69/76 is a recurring "decimal" in both base 10 and base 2.

    • decimal: 0.90(789473684210526315)
    • binary: 0.11(101000011010111100)

    So the result gets rounded off, and multiplying by the denominator may not result in the orginal numerator. A more commonly-cited example of this situation is 1/3*3 = 0.33333333*3 = 0.99999999.

    That the double version gives the exact answer is just a coincidence. The rounding error in the multiplication just happens to cancel out the rounding error in the division.

    If this result is confusing, it may be because you've heard that "double has rounding errors and decimal is exact". But decimal is only exact at representing decimal fractions like 0.1 (which is 0.0 0011 0011... in binary). When you have a factor of 19 in the denominator, it doesn't help you.

提交回复
热议问题