acos(1) returns NaN for some values, not others

Question

I have a list of latitude and longitude values, and I\'m trying to find the distance between them. Using a standard great circle method, I need to find:

aco         


        

Answer 1

Can't tell exactly without seeing your data (try dput), but this is mostly likely a consequence of FAQ 7.31.

(x1 <- 1)
## [1] 1
(x2 <- 1+1e-16)
## [1] 1
(x3 <- 1+1e-8)
## [1] 1
acos(x1)
## [1] 0
acos(x2)
## [1] 0
acos(x3)
## [1] NaN

That is, even if your values are so similar that their printed representations are the same, they may still differ: some will be within .Machine$double.eps and others won't ...

One way to make sure the input values are bounded by [-1,1] is to use pmax and pmin: acos(pmin(pmax(x,-1.0),1.0))

Answer 2

A simple workaround is to use pmin(), like this:

acos(pmin(sin(lat1)*sin(lat2) + cos(lat1)*cos(lat2) * cos(long2-long1),1))

It now ensures that the precision loss leads to a value no higher than exactly 1.

This doesn't explain what is happening, however.

(Edit: Matthew Lundberg pointed out I need to use pmin to get it tow work with vectorized inputs. This fixes the problem with getting it to work, but I'm still not sure why it is rounding incorrectly.)