Why does Excel not round according to 8-byte IEEE 754

后端 未结 1 627
遇见更好的自我
遇见更好的自我 2020-12-03 19:22

The following expression evaluates to false in C#:

(1 + 1 + 0.85) / 3 <= 0.95

And I suppose it does so in most other progra

相关标签:
1条回答
  • 2020-12-03 19:58

    The article that you linked to is explicit about doing something nonstandard with values near 0:

    Example when a value reaches zero 1.In Excel 95 or earlier, enter the following into a new workbook: A1: =1.333+1.225-1.333-1.225

    2.Right-click cell A1, and then click Format Cells. On the Number tab, click Scientific under Category. Set the Decimal places to 15. Instead of displaying 0, Excel 95 displays -2.22044604925031E-16.

    Excel 97, however, introduced an optimization that attempts to correct for this problem. Should an addition or subtraction operation result in a value at or very close to zero, Excel 97 and later will compensate for any error introduced as a result of converting an operand to and from binary.

    The unspecified "optimization that attempts to correct for this problem" does mean that caution should be used in using Excel for numerical computations when strict agreement with IEEE 754 is required. Perhaps using VBA (which is unlikely to have this "optimization"?) might be a workaround.

    0 讨论(0)
提交回复
热议问题