Addition of Double values inconsistent

后端 未结 7 1998
渐次进展
渐次进展 2021-01-20 01:13

I came across following issue while developing some engineering rule value engine using eval(...) implementation.

    Dim first As Double = 1.1
    Dim secon         


        
7条回答
  •  再見小時候
    2021-01-20 01:40

    From the documentaiton: http://msdn.microsoft.com/en-us/library/system.double.aspx

    Floating-Point Values and Loss of Precision

    Remember that a floating-point number can only approximate a decimal number, and that the precision of a floating-point number determines how accurately that number approximates a decimal number. By default, a Double value contains 15 decimal digits of precision, although a maximum of 17 digits is maintained internally. The precision of a floating-point number has several consequences:

    Two floating-point numbers that appear equal for a particular precision might not compare equal because their least significant digits are different.

    A mathematical or comparison operation that uses a floating-point number might not yield the same result if a decimal number is used because the floating-point number might not exactly approximate the decimal number.

    A value might not roundtrip if a floating-point number is involved. A value is said to roundtrip if an operation converts an original floating-point number to another form, an inverse operation transforms the converted form back to a floating-point number, and the final floating-point number is equal to the original floating-point number. The roundtrip might fail because one or more least significant digits are lost or changed in a conversion.

    In addition, the result of arithmetic and assignment operations with Double values may differ slightly by platform because of the loss of precision of the Double type. For example, the result of assigning a literal Double value may differ in the 32-bit and 64-bit versions of the .NET Framework. The following example illustrates this difference when the literal value -4.42330604244772E-305 and a variable whose value is -4.42330604244772E-305 are assigned to a Double variable. Note that the result of the Parse(String) method in this case does not suffer from a loss of precision.

提交回复
热议问题