integrate() in R gives terribly wrong answer

Question

I was trying to integrate the following function from -infinity to infinity. The answer should be 0.2, but R gives a ridiculously small number. What\'s wrong?

Answer 1

I'll need a bit longer to explain this fully, and hopefully other users will add to this wiki.

From ?integrate, the abs.tol argument is defined as

absolute accuracy requested.

And further down is the following note:

When integrating over infinite intervals do so explicitly, rather than just using a large number as the endpoint. This increases the chance of a correct answer – any function whose integral over an infinite interval is finite must be near zero for most of that interval.

So if you want absolute accuracy as opposed to relative accuracy (which is defined as the result from .Machine$double.eps^0.25) then you can do

> integrate(f, Inf, -Inf, abs.tol = 0L)
0.2 with absolute error < 8.4e-06

The default argument for abs.tol is passed from rel.tol, which is .Machine$double.eps^0.25

Let's see what goes on "inside" a bit.

ifoo<-integrate(f,-Inf,Inf,abs.tol=1e-20)
5.275825e-21 with absolute error < 9.8e-21
str(ifoo)
List of 5
 $ value       : num 5.28e-21
 $ abs.error   : num 9.81e-21
 $ subdivisions: int 3
 $ message     : chr "OK"
 $ call        : language integrate(f = f, lower = -Inf, upper = Inf, abs.tol = 1e-20)
 - attr(*, "class")= chr "integrate"

ifoo<-integrate(f,-Inf,Inf,abs.tol=1e-40)
0.2 with absolute error < 8.4e-06
str(ifoo)
List of 5
 $ value       : num 0.2
 $ abs.error   : num 8.36e-06
 $ subdivisions: int 21
 $ message     : chr "OK"
 $ call        : language integrate(f = f, lower = -Inf, upper = Inf, abs.tol = 1e-40)
 - attr(*, "class")= chr "integrate"

Notice the sudden jump in the number of subdivisions. In general, more subdivisions means better accuracy, which after all is the point of Calculus: reduce the subdivision width to nothing to get the exact answer. My guess is that, with a large(ish) abs.tol, it only takes a few subdivisions for the calculated value to agree with some 'estimated tolerance error' , but when the required tolerance gets small enough, more subdivisions are "added."

Edit: with thanks to Hong Ooi, who actually looked at the integrand in question. :-) . Because this function has a cusp at x==25, i.e. a discontinuity in the derivative, the optimization algorithm likely gets "misled" about convergence. Oddly enough, by taking advantage of the fact that this integrand goes to near-zero very quickly, the result is better when not integrating out to +/-Inf . In fact:

Rgames> integrate(f,20,30)
0.2 with absolute error < 1.9e-06
Rgames> integrate(f,22,27)
0.2 with absolute error < 8.3e-07
Rgames> integrate(f,0,50)
0.2 with absolute error < 7.8e-05