Apparent F#/BCL floating point bug

前端 未结 3 1715
清酒与你
清酒与你 2021-01-21 11:17

The following is in FSI:

> System.Math.Round(0.2916, 2);;
val it : float = 0.29
> it * 100.;;
val it : float = 29.0
> int it;;
val it : int = 28
         


        
3条回答
  •  执念已碎
    2021-01-21 11:46

    You can change the F# FSI Pretty Printer like Matlab "format long"

    fsi.AddPrinter( fun (x:float) -> sprintf "%26.16e" x);;
    

    FSI output

    System.Math.Round( 0.2916, 2)   //  2.8999999999999998e-001
    
    1./3.                           // 3.3333333333333331e-001
    

提交回复
热议问题