Issue with precision of Ruby math operations

后端 未结 3 504
心在旅途
心在旅途 2021-01-23 22:18

Do you know how to fix the following issue with math precision?

p RUBY_VERSION # => \"1.9.1\"
p 0.1%1 # => 0.1
p 1.1%1 # => 0.1
p 90.0%1 # => 0.0
p 9         


        
3条回答
  •  盖世英雄少女心
    2021-01-23 22:47

    Big Decimal


    As the man said;

    Squeezing infinitely many real numbers into a finite number of bits requires an approximate representation.


    I have however had great success using the BigDecimal class. To quote its intro

    Ruby provides built-in support for arbitrary precision integer arithmetic. For example:

    42**13 -> 1265437718438866624512

    BigDecimal provides similar support for very large or very accurate floating point numbers.


    Taking one of your examples;

    >> x = BigDecimal.new('900.1')
    => #
    >> x % 1
    => #
    >> y = x % 1
    => #
    >> y.to_s
    => "0.1E0"
    >> y.to_f
    => 0.1
    


    As you can see, ensuring decent precision is possible but it requires a little bit of effort.

提交回复
热议问题