How can you perform this improper integral as Mathematica does?

Question

Take this Mathematica code:

f[x_] := Exp[-x];
c = 0.9;
g[x_] := c*x^(c - 1)*Exp[-x^c];
SetPrecision[Integrate[f[x]*Log         


        

Answer 1

A very interesting problem.

First note that the integrand

from numpy import exp

def f(x):
    return exp(-x) 

def g(x):
    c = 0.9
    return c * x**(c - 1) * exp(-x ** c)

def integrand(x):
    return f(x) * log(f(x) / g(x))

has a singularity at 0 that is integrable, and the integral over [0, infty] can be evaluated analytically. After some manipulation, you'll find

import numpy
import scipy.special

c = 0.9

# euler_mascheroni constant
gamma = 0.57721566490153286060
val = scipy.special.gamma(c + 1) - 1 - numpy.log(c) + (c - 1) * gamma

print(val)

0.0094047810750603

wolfram-alpha gives its value correctly to many digits. To reproduce this with numerical methods, a good first try is always tanh-sinh quadrature (e.g., from quadpy, a project of mine). Cut off the domain at some large value, where the function is almost 0 anyway, then:

from numpy import exp, log
import quadpy


def f(x):
    return exp(-x)


def g(x):
    c = 0.9
    return c * x**(c - 1) * exp(-x ** c)


def integrand(x):
    return f(x) * log(f(x) / g(x))


val, err = quadpy.tanh_sinh(integrand, 0.0, 100.0, 1.0e-8)
print(val)

0.009404781075063085

---

Now for some other things that, perhaps surprisingly, do not work so well.

When seeing an integral of the type exp(-x) * f(x), the first thing that should come to mind is Gauss-Laguerre quadrature. For example with quadpy (one of my projects):

import numpy
import quadpy

c = 0.9


def f(x):
    return numpy.exp(-x)


def g(x):
    return c * x ** (c - 1) * numpy.exp(-x ** c)


scheme = quadpy.e1r.gauss_laguerre(100)
val = scheme.integrate(lambda x: numpy.log(f(x) / g(x)))

print(val[0])

This gives

0.010039543105755215

which is a surprisingly bad approximation for the actual value despite the fact that we were using 100 integration points. This is due to the fact that the integrand cannot be approximated very well by polynomials, especially the terms log(x) and x ** c:

import numpy
from numpy import exp, log, ones
from scipy.special import gamma
import quadpy


c = 0.9


def integrand(x):
    return exp(-x) * (-x - log(c) - (c - 1) * log(x) - (-x ** c))


scheme = quadpy.e1r.gauss_laguerre(200)
val = scheme.integrate(lambda x: -x - log(c) - (c - 1) * log(x) - (-x ** c))[0]

vals = numpy.array([
    - scheme.integrate(lambda x: x)[0],
    -log(c) * scheme.integrate(lambda x: ones(x.shape))[0],
    -(c - 1) * scheme.integrate(lambda x: log(x))[0],
    scheme.integrate(lambda x: x ** c)[0]
])
euler_mascheroni = 0.57721566490153286060
exact = numpy.array([
    -1.0,
    -log(c),
    euler_mascheroni * (c-1),
    gamma(c + 1)
])
print("approximation, exact, diff:")
print(numpy.column_stack([vals, exact, abs(vals - exact)]))
print()
print("sum:")
print(sum(vals))

approximation, exact, diff:
[[-1.00000000e+00 -1.00000000e+00  8.88178420e-16]
 [ 1.05360516e-01  1.05360516e-01  6.93889390e-17]
 [-5.70908293e-02 -5.77215665e-02  6.30737142e-04]
 [ 9.61769857e-01  9.61765832e-01  4.02488825e-06]]

sum:
0.010039543105755278

Answer 2

In julia, the QuadGK package can do these integrals. Just doing this directly you will bump into issues, as you note:

f(x) = exp(-x)
g(x; c=0.9) = c*x^(c - 1)*exp(-x^c)
h(x) = f(x) * log(f(x)/g(x))
using QuadGK
a,b = 0.001, Inf
quadgk(h, a, b)  # errors

But expanding the log(f/g) to log(f) - (log(c) + (c-1)log(x) + x^c) we can get each term to integrate:

c = 0.9
quadgk(x -> f(x) * -x, a,b)
quadgk(x -> -f(x)*log(c), a,b)
quadgk(x -> -f(x)*(c-1)*log(x), a,b)
quadgk(x -> f(x) * x^c, a,b)

Adding up the values gives the answer.

You can also get the answer by filtering out the NaN values, which may be much more inefficient:

h1(x) = isnan(h(x)) ? 0.0 : h(x)
quadgk(h1, a,b) # (0.010089328699390816, 9.110982026738999e-11)

Using big(a) and big(b) can get you more decimal points.