rounding-error | 易学教程

Round-off error with SVG linear gradient in Chrome?

阅读更多关于 Round-off error with SVG linear gradient in Chrome?

问题 I'm seeing some issues recently with Chrome to render SVG linear gradient. Have a look at this SVG below: <!DOCTYPE html> <html lang="en" xmlns="http://www.w3.org/1999/xhtml"> <body> <svg> <g> <linearGradient id="linearGradient" gradientUnits="userSpaceOnUse" x1="79px" x2="706px" y1="0" y2="0"> <stop class="" offset="54.54545454545455%" style="stop-color: red;"></stop> <stop class="" offset="63.63636363636364%" style="stop-color: red;"></stop> <stop class="" offset="63.63636363636364%" style=

Is there a datatype “Decimal” in R?

阅读更多关于 Is there a datatype “Decimal” in R?

问题 I read data stored in the format DECIMAL from a MySQL-Table. I want to do calculations on those numbers within R. I used to cast them to a numeri represaentation using as.numeric() , but the documentation says: numeric is identical to double (and real). But is there also a datatype Decimal in R? (Datatype without rounding errors,...) Here a simple example for the problem with rounding errors: numbersStrings = c("0.1", "0.9") numbersNumeric = as.numeric(numbersStrings) numbersMirror = c

Handling doubles with large integer parts in C++

阅读更多关于 Handling doubles with large integer parts in C++

问题 I need to handle sub-metre coordinates in C++ which have large integer parts (e.g. 515876.12 // 5117789.22), but I'm having issues with rounding: double inUTMX = 560351.12 is displayed as 560351 double inUTMY = 5113570.22 is displayed as 5.11357e+06 I can normalise the coordinates for processing if necessary (e.g. /1e5), but I need to read-in the sub-metre coordinates via command line in the first place. Trouble is they always get rounded. Is there a neat way to deal with doubles that have

When does floating-point rounding-errors occur? [duplicate]

阅读更多关于 When does floating-point rounding-errors occur? [duplicate]

问题 This question already has answers here : Is floating point math broken? (31 answers) Closed 4 years ago . As I was debugging my VBA code, I came across this weird phenomenon: This loop Dim x,y as Double x = 0.7 y = 0.1 For x = x - y To x + y Step y Next x runs only twice! I tried many variations of this code to nail down the problem, and here is what I came up with: Replacing the loop boundaries with simple numbers (0.6 to 0.8) - helped. Replacing variables with numbers (all the combinations)

Why adding from biggest to smallest floating-point numbers is less accurate than adding from smallest to biggest?

阅读更多关于 Why adding from biggest to smallest floating-point numbers is less accurate than adding from smallest to biggest?

问题 My Java textbook states that adding from biggest to smallest is less accurate than adding from smallest to biggest when dealing with floating-point numbers. However, he doesn't go on to clearly explain why this is the case. 回答1: Floating point has a limited number of digits of precision (6 for float , 15 for double ). The calculation 1.0e20d + 1 gives the result 1.0e20 because there is not enough precision to represent the number 100,000,000,000,000,000,001 If you start with the largest

Javascript. Increase or decrease float as little as possible

阅读更多关于 Javascript. Increase or decrease float as little as possible

问题 In javascipt I have av floating point "a" like this: var a = 5.; Now I want a new number "b" that is barely larger than "a". I could do this: var b = a + 1.e-10; But what if "a" is a really small number? var a = 5.e-20; var b = a + 1.e-10; Now "b" is many orders of magnitude larger than "a". Also if I make the difference between "a" and "b" too small, a large "a" could cause the difference to be rounded off. How do I make the number "b" larger than any number "a", but closer to "a" than any

Haskell logbase error

阅读更多关于 Haskell logbase error

问题 I am trying to calculate the length of an Integer in Haskell, using the fact that the length is equal to truncate (log10(x)+1) . Using Integers I created: len :: Integer -> Integer len i = toInteger (truncate (logBase 10 (fromIntegral i)) + 1) Unfortunately, not all numbers get the correct length. I tried a few different cases and found that: logBase 10 10 = 1.0 logBase 10 100 = 2.0 logBase 10 1000 = 2.9999..6 logBase 10 10000 = 4.0 logBase 10 100000 = 5.0 logBase 10 1000000 = 5.9999999 Is

Why do I get platform-specific result for std::exp? [duplicate]

阅读更多关于 Why do I get platform-specific result for std::exp? [duplicate]

问题 This question already has answers here : Is floating point math broken? (31 answers) Math precision requirements of C and C++ standard (1 answer) Closed 11 months ago . I have a program that were giving slithly different results under Android and Windows. As I validate the output data against a binary file containign expected result, the difference, even if very small (rounding issue) is annoying and I must find a way to fix it. Here is a sample program: #include <iostream> #include <iomanip>

Numpy rounds in a different way than python

阅读更多关于 Numpy rounds in a different way than python

问题 The code import numpy as np a = 5.92270987499999979065 print(round(a, 8)) print(round(np.float64(a), 8)) gives 5.92270987 5.92270988 Any idea why? Found nothing relevant in numpy sources. Update: I know that the proper way to deal with this problem is to construct programs in such a way that this difference is irrelevant. Which I do. I stumbled into it in regression testing. Update2: Regarding the @VikasDamodar comment. One shouldn't trust the repr() function: >>> np.float64(5

Preventing R From Rounding

阅读更多关于 Preventing R From Rounding

问题 How do I prevent R from rounding? For example, > a<-893893084082902 > a [1] 8.93893e+14 I am losing a lot of information there. I have tried signif() and it doesn't seem to do what I want. Thanks in advance! (This came up as a result of a student of mine trying to determine how long it would take to count to a quadrillion at a number per second) 回答1: It's not rounding; it's just the default format for printing large (or small) numbers. a <- 893893084082902 > sprintf("%f",a) [1]