You've discovered the wonders of floating point arithmetic. Since the internal representation of a number is in binary, certain decimal numbers can't be represented precisely. As a result, some arithmetic operations will give unexpected results in the least significant digits.
This is covered in exhausting detail in, for example, numerical methods courses, but the Wikipedia article isn't bad.
