floating-point-precision

double and float comparison [duplicate]

本小妞迷上赌 提交于 2019-12-02 08:54:31
问题 This question already has answers here : Comparing float and double (3 answers) Closed 4 years ago . According to this post, when comparing a float and a double, the float should be treated as double. The following program, does not seem to follow this statement. The behaviour looks quite unpredictable. Here is my program: void main(void) { double a = 1.1; // 1.5 float b = 1.1; // 1.5 printf("%X %X\n", a, b); if ( a == b) cout << "success " <<endl; else cout << "fail" <<endl; } When I run the

Why does using modulo on non-integer values lose floating-point precision? [duplicate]

旧城冷巷雨未停 提交于 2019-12-02 08:14:02
This question already has an answer here: Why do I see a double variable initialized to some value like 21.4 as 21.399999618530273? 14 answers I am wondering why I am losing precision when using this code : double x = 12.0456; // or float : same result System.out.println(x); // outputs 12.0456 obviously x %= 1; // should now be equal to 0.0456 right? System.out.println(x); // outputs 0.04560000000000031 or 0.045599937 when using float 12.0456 modulo 1 should equal 0.0456 right? But it shows a slightly different value, why do I keep losing precision? I mean the code should substract exactly 1

double and float comparison [duplicate]

瘦欲@ 提交于 2019-12-02 04:44:24
This question already has an answer here: Comparing float and double 3 answers According to this post , when comparing a float and a double, the float should be treated as double. The following program, does not seem to follow this statement. The behaviour looks quite unpredictable. Here is my program: void main(void) { double a = 1.1; // 1.5 float b = 1.1; // 1.5 printf("%X %X\n", a, b); if ( a == b) cout << "success " <<endl; else cout << "fail" <<endl; } When I run the following program, I get "fail" displayed. However, when I change a and b to 1.5, it displays "success". I have also

floating point operations in go

谁都会走 提交于 2019-12-01 21:40:53
问题 Here's the sample code in go: package main import "fmt" func mult32(a, b float32) float32 { return a*b } func mult64(a, b float64) float64 { return a*b } func main() { fmt.Println(3*4.3) // A1, 12.9 fmt.Println(mult32(3, 4.3)) // B1, 12.900001 fmt.Println(mult64(3, 4.3)) // C1, 12.899999999999999 fmt.Println(12.9 - 3*4.3) // A2, 1.8033161362862765e-130 fmt.Println(12.9 - mult32(3, 4.3)) // B2, -9.536743e-07 fmt.Println(12.9 - mult64(3, 4.3)) // C2, 1.7763568394002505e-15 fmt.Println(12.9 - 3

Any risk of using float variables as loop counters and their fractional increment/decrement for non “==” conditions?

拥有回忆 提交于 2019-12-01 18:43:19
问题 Are we safe to use floats as loop-counters and to increment/decrement them by fractional amounts at each iteration,like in the seemingly risk-free program below?Of course I know that using floats as operands for the == operator is a dumb thing to do.But what's wrong with using floats as operands for other comparison operations for "normal" purposes? By "normal" I mean that,well,even though floats may not be the exact numerical representation of the number,but isn't a variation like 0

Any risk of using float variables as loop counters and their fractional increment/decrement for non “==” conditions?

对着背影说爱祢 提交于 2019-12-01 18:26:40
Are we safe to use floats as loop-counters and to increment/decrement them by fractional amounts at each iteration,like in the seemingly risk-free program below?Of course I know that using floats as operands for the == operator is a dumb thing to do.But what's wrong with using floats as operands for other comparison operations for "normal" purposes? By "normal" I mean that,well,even though floats may not be the exact numerical representation of the number,but isn't a variation like 0.000000001 irrelevant and can be ignored in most cases? (For example in the following program that isn't even

Error With Using (int) and (double) together to Cut off Decimals

我是研究僧i 提交于 2019-12-01 18:20:01
问题 When I am using (int) with (double) some times it is not working correct. Look At The PHP Code Example: I Need To LEAVE 2 Decimals And REMOVE Other... I Know number_format(); function But I Cannot Use It. Because It Is Rounding Number number_format(24.299,2); Output: 24.30 I Need: 24.29 <?php $str="158.2"; echo (double)$str; // Output: 158.2 echo (double)$str*100; // Output: 15820 echo (int)((double)$str*100); // Output: 15819 <-WHY? It Must To Be 15820, Why 15819? echo ((int)((double)$str

Error With Using (int) and (double) together to Cut off Decimals

让人想犯罪 __ 提交于 2019-12-01 18:00:15
When I am using (int) with (double) some times it is not working correct. Look At The PHP Code Example: I Need To LEAVE 2 Decimals And REMOVE Other... I Know number_format(); function But I Cannot Use It. Because It Is Rounding Number number_format(24.299,2); Output: 24.30 I Need: 24.29 <?php $str="158.2"; echo (double)$str; // Output: 158.2 echo (double)$str*100; // Output: 15820 echo (int)((double)$str*100); // Output: 15819 <-WHY? It Must To Be 15820, Why 15819? echo ((int)((double)$str*100)/100); // Output: 158.19 ?> I need To leave two decimals in the number and cut other WITHOUT rounding

Switching between float and double precision at compile time

两盒软妹~` 提交于 2019-12-01 08:19:50
Where should I look at if I want to switch between float and double precision at compile time. Its like, if user wants everything in float instead of double precision how I can maintain this flexibility? In other words, how should I define a variable that could be either float or double precision conditionally? If it is OK to make the switch at compile time, a simple typedef would do: #ifdef USE_DOUBLES typedef double user_data_t; #else typedef float user_data_t; #endif Use user_data_t in your code, and set USE_DOUBLES if you want doubles at compile time: g++ -DUSE_DOUBLES=1 myprogram.cpp

How to actually avoid floating point errors when you need to use float?

假装没事ソ 提交于 2019-12-01 06:52:05
I am trying to affect the translation of a 3D model using some UI buttons to shift the position by 0.1 or -0.1. My model position is a three dimensional float so simply adding 0.1f to one of the values causes obvious rounding errors. While I can use something like BigDecimal to retain precision, I still have to convert it from a float and back to a float at the end and it always results in silly numbers that are making my UI look like a mess. I could just pretty the displayed values but the rounding errors will only get worse with more editing and they make my save files rather hard to read.