Avoid overflow with softplus function in python

Question

I am trying to implement the following softplus function:

log(1 + exp(x))

I\'ve tried it with math/numpy and float64 as data type, but whenever

Answer 1

There is a relation which one can use:

log(1+exp(x)) = log(1+exp(x)) - log(exp(x)) + x = log(1+exp(-x)) + x

So a safe implementation, as well as mathematically sound, would be:

log(1+exp(-abs(x))) + max(x,0)

This works both for math and numpy functions (use e.g.: np.log, np.exp, np.abs, np.maximum).