Why don't operations on double-precision values give expected results?

时间秒杀一切 提交于 2019-11-29 11:18:32

The do give the expected results. Your expectations are incorrect.

When you type the double-precision literal 2.14656, what you actually get is the closest double-precision value, which is:

2.14656000000000002359001882723532617092132568359375

the println happens to round this when it prints it out (to 17 significant digits), so you see the nice value that you expect.

After the modulus operation (which is exact), the value is:

0.14656000000000002359001882723532617092132568359375

Again, this is rounded when it gets printed, but because there is one less leading digit, the round point is one digit farther to the right, and so you see that trailing 2.

A base-2 system is being used to store a base-10 number of arbitrary precision.

When using integer binary, the "round" numbers are 2, 4, 8, 16, 32, 64, etc. You'll note that none of these are 10, 100, 1000, 10000, or anything that would be considered a "round" number in base-10.

Float and double values were constructed for fast operations within a CPU. Math between multiple numbers can be quickly done, even if you're dividing 1.42E-13 by 2.11E47. They were not constructed to provide "round" numbers in base-10. This is the side-effect of that decision.

When using floating point numbers, you have to accept that this will happen. It's not 100% precise, but it is accurate otherwise. So you should never use floating point variables where you need 100% precision. Where you need to do this, find a way to store it in an int value. It's also a good idea, where you do use a floating point number, to round the output to a number of decimal places to avoid the occasional display like you have there.

For example, I work in the financial industry, and we represent market prices as a number of ticks rather than an actual price. When we need to display a price, we multiply this value by the tick size, 1/100, 1/32, or 1/20, etc, and then round to the appropriate number of decimal places. The number of ticks, the numerator and denominator of the tick size, are all stored as integers in our databases. We don't actually store floating point values for anything except for calculated values like moving averages.

易学教程内所有资源均来自网络或用户发布的内容,如有违反法律规定的内容欢迎反馈
该文章没有解决你所遇到的问题?点击提问,说说你的问题,让更多的人一起探讨吧!