Unexpected loss of precision when dividing doubles

前端 未结 8 1968
慢半拍i
慢半拍i 2021-02-09 14:33

I have a function getSlope which takes as parameters 4 doubles and returns another double calculated using this given parameters in the following way:

double QSw         


        
8条回答
  •  小蘑菇
    小蘑菇 (楼主)
    2021-02-09 14:58

    The problem here is that (c-a) is small, so the rounding errors inherent in floating point operations is magnified in this example. A general solution is to rework your equation so that you're not dividing by a small number, I'm not sure how you would do it here though.

    EDIT:

    Neil is right in his comment to this question, I computed the answer in VB using Doubles and got the same answer as mathematica.

提交回复
热议问题