Why are my BigDecimal objects initialized with unexpected rounding errors?

落爺英雄遲暮 提交于 2019-11-28 23:14:30

Initialize with Strings Instead of Floats

In general, you can't get reliable behavior with Floats. You're making the mistake of initializing your BigDecimals with Float values instead of String values, which introduces some imprecision right at the beginning. For example, on my 64-bit system:

float1 = 34.13985572755337
float2 = 34.13985572755338

# You can use string literals here, too, if your Float can't be properly
# represented. For example:
#
#    BigDecimal.new("34.13985572755337", 9)
#
# would be safer, but Float#to_s works fine with the provided corpus.
bd1 = BigDecimal.new(float1.to_s, 9)
bd2 = BigDecimal.new(float2.to_s, 9)

bd1.to_s
#=> "0.3413985572755337E2"
bd2.to_s
#=> "0.3413985572755338E2"

bd1.to_f == float1
#=> true
bd2.to_f == float2
#=> true

This is one of those cases where the internal representation of the arguments matter. Therefore, your mileage will vary depending on how you initialize your object.

易学教程内所有资源均来自网络或用户发布的内容,如有违反法律规定的内容欢迎反馈
该文章没有解决你所遇到的问题?点击提问,说说你的问题,让更多的人一起探讨吧!