floating-point-precision

问题 Got a question regarding to the underlying data structure of float (and precision) in Python: >>> b = 1.4 + 2.3 >>> b 3.6999999999999997 >>> c = 3.7 >>> c 3.7000000000000002 >>> print b, c 3.7 3.7 >>> b == c False it seems the values of b and c are machine dependent, they are the numbers that closest to the target values but not exactly the same numbers. I was supervised that we get the 'right' numbers with 'Print', and someone told me that it was because print 'lies' while Python chose to

问题 I was working on a project to compute the Leibniz approximation for pi with the below code: def pi(precision): sign = True ret = 0 for i in range(1,precision+1): odd = 2 * i - 1 if sign: ret += 1.0 / odd else: ret -= 1.0 / odd sign = not sign return ret However, the output value was always was 12 digits long. How can I increase the precision (e.g. more digits) of the calculation? Does Python support more precise floating points, or will I have to use some external library? 回答1: Try using

问题 I was delving into multi-precision arithmetics, and there is a nice fast class of algorithms, described in Jonathan Richard Shewchuk, "Adaptive Precision Floating-Point Arithmetic and Fast Robust Geometric Predicates", 1997, Discrete & Computational Geometry, pages: 305–363. However, these algorithms rely on the FPU using round-to-even tiebreaking. On CPU, it would be easy, one would just check or set the FPU state word and would be sure. However, there is no such instruction (yet?) for GPU

问题 I've found this problem in many interview exams, but don't see how to work out the proper solution myself. The problem is: How many digits of accuracy can be represented by a floating point number represented by two 16-bit words? The solution is apparently approximately 6 digits. Where does this come from, and how would you work it out? 回答1: It's quite simple: a 32 bit IEEE-754 float has 23+1 bits for the mantissa (AKA significand, in IEEE-speak). The size of the mantissa more or less

问题 I'm writing an extension method to compare two floats using a set number of decimal points (significant figures) to determine if they are equal instead of a tolerance or percentage difference. Looking through the other questions regarding float comparison I see complex implementations. Have I oversimplified or is this valid? /// <summary> /// Determines if the float value is equal to (==) the float parameter according to the defined precision. /// </summary> /// <param name="float1">The

问题 Strange output when I use float instead of double #include <stdio.h> void main() { double p,p1,cost,cost1=30; for (p = 0.1; p < 10;p=p+0.1) { cost = 30-6*p+p*p; if (cost<cost1) { cost1=cost; p1=p; } else { break; } printf("%lf\t%lf\n",p,cost); } printf("%lf\t%lf\n",p1,cost1); } Gives output as expected at p = 3; But when I use float the output is a little weird. #include <stdio.h> void main() { float p,p1,cost,cost1=40; for (p = 0.1; p < 10;p=p+0.1) { cost = 30-6*p+p*p; if (cost<cost1) {

问题 I am writing a loop that increments with a float, but I have come across a floating-point arithmetic issue illustrated in the following example: for(float value = -2.0; value <= 2.0; value += 0.2) std::cout << value << std::endl; Here is the output: -2 -1.8 -1.6 -1.4 -1.2 -1 -0.8 -0.6 -0.4 -0.2 1.46031e-07 0.2 0.4 0.6 0.8 1 1.2 1.4 1.6 1.8 Why exactly am I getting 1.46031e-07 instead of 0 ? I know this has something to do with floating-point errors, but I can't grasp why it is happening and

问题 So I'm trying to convert a float DataFrame to a list (of list) by using row.values.tolist() ( row was read from a CSV file). It does the job pretty okay though for a few values the precision is being affected, so, for instance, instead of 32.337 it's outputting 32.336999999999996. Since tolist() yields a list of list and I need to work with lists , I decided to switch to list(row.values.flatten()) , but it introduces precision issues for almost any value in the list, which just makes it worst

问题 I'm using float 24 bit to store a floating point value in a compiler MRK III from NXP. It stores the 24 bit float value as 3 byte Hex in Data memory. Now when I'm using IEEE 754 float point conversion to retrieve the number back from binary to real, I'm getting something very strange. Let me put it this way with an example - Note - "since my compiler supports float 24 bit (along with float 32), I'm assigning value something like this." Sample Program : float24 f24test; float f32test; f32test=

问题 This question already has answers here : Why do I see a double variable initialized to some value like 21.4 as 21.399999618530273? (14 answers) Closed 6 years ago . I am wondering why I am losing precision when using this code : double x = 12.0456; // or float : same result System.out.println(x); // outputs 12.0456 obviously x %= 1; // should now be equal to 0.0456 right? System.out.println(x); // outputs 0.04560000000000031 or 0.045599937 when using float 12.0456 modulo 1 should equal 0.0456