Why is my number being rounded incorrectly?

Question

This feels like the kind of code that only fails in-situ, but I will attempt to adapt it into a code snippet that represents what I\'m seeing.

float f = myFl         


        

Answer 1

I'd like to offer a different explanation.

Here's the code, which I've annotated (I looked into memory to dissect the floats):

 float myFloat = 13.45; //In binary is 1101.01110011001100110011
 int myConstInt = 20;
 float f = myFloat * myConstInt; //In binary is exactly 100001101 (269 decimal)
 int i = (int)f; // Turns float 269 into int 269 -- no surprises
 int i2 = (int)(myFloat * myConstInt);//"Extra precision" causes round to 268

Let's look closer at the calculations:

• f = 1101.01110011001100110011 * 10100 = 100001100.111111111111111 111

  The part after the space is bits 25-27, which cause bit 24 to be rounded up, and hence the whole value to be rounded up to 269

• int i2 = (int)(myFloat * myConstInt)

  myfloat is extended to double precision for the calculation (0s are appended): 1101.0111001100110011001100000000000000000000000000000

  myfloat * 20 = 100001100.11111111111111111100000000000000000000000000

  Bits 54 and beyond are 0s, so no rounding is done: the cast results in the integer 268.

  (A similar explanation would work if extended precision is used.)

UPDATE: I refined my answer and wrote a full-blown article called When Floats Don’t Behave Like Floats