c++ floating point precision loss: 3015/0.00025298219406977296

Question

The problem.

Microsoft Visual C++ 2005 compiler, 32bit windows xp sp3, amd 64 x2 cpu.

Code:

double a = 3015.0; 
double b = 0.00025298219406977296         


        

Answer 1

Are you sure you're examining the value of f right after the fstp instruction? If you've got optimizations turned on perhaps the watch window could be showing a value taken at some later point (this seems a bit plausible as you say you're looking at the fractional part of f later - does some instruction wind up masking it out somehow?)