Unexpected loss of precision when dividing doubles

前端 未结 8 1969
慢半拍i
慢半拍i 2021-02-09 14:33

I have a function getSlope which takes as parameters 4 doubles and returns another double calculated using this given parameters in the following way:

double QSw         


        
8条回答
  •  终归单人心
    2021-02-09 15:00

    Could it be that you use DirectX or OpenGL in your project? If so they can turn off double precision and you will get strange results.

    You can check your precision settings with

    std::sqrt(x) * std::sqrt(x)
    

    The result has to be pretty close to x. I met this problem long time ago and spend a month checking all the formulas. But then I've found

    D3DCREATE_FPU_PRESERVE
    

提交回复
热议问题