问题
The following is in FSI:
> System.Math.Round(0.2916, 2);;
val it : float = 0.29
> it * 100.;;
val it : float = 29.0
> int it;;
val it : int = 28
The result is the same everywhere I tried - a compiled F# 3.1/.NET 4.0 application, FSI in Visual Studio 2013 and 2015, .NET Fiddle, @fsibot....
Surely this is a bug somewhere, isn't it? What's going on here?
回答1:
This is just how floating point numbers work. The number that appears as 29 in the output is actually slightly smaller than 29 (because floating point numbers are not precise):
> (System.Math.Round(0.2916, 2) * 100.0) - 29.0;;
val it : float = -3.552713679e-15
回答2:
float
is binary floating point data type. Not all decimal floating point values can be represented exactly as binary floating point. What FSI shows as 29
is actually 28.999999999999996
.
If you want to store precise decimal floating point values, use decimal
data type (to make a number literal decimal
instead of float
add m
to it - e.g. 0.2916m
).
> System.Math.Round(0.2916m, 2);;
val it : decimal = 0.29M
> it * 100m;;
val it : decimal = 29.00M
> int it;;
val it : int = 29
回答3:
You can change the F# FSI Pretty Printer like Matlab "format long"
fsi.AddPrinter( fun (x:float) -> sprintf "%26.16e" x);;
FSI output
System.Math.Round( 0.2916, 2) // 2.8999999999999998e-001
1./3. // 3.3333333333333331e-001
来源:https://stackoverflow.com/questions/33271337/apparent-f-bcl-floating-point-bug