Unwanted [Nan] output in Python neural network

Question

newbie here. Just switched over from JS to Python to build Neural nets but getting [Nan] outputs from it.

The weird thing is that my sigmoid func. doesn\'t seem to e

Answer 1

There are at least two issues with your code.

The first is the inexplicable use of 2 return statements in your sigmoid function, which should simply be:

def sigmoid(x):
  return 1/(1 + np.exp(-x))

which gives the correct result for x=0 (0.5), and goes to 1 for large x:

sigmoid(0)
# 0.5
sigmoid(20)
# 0.99999999793884631

Your (wrong) sigmoid:

def your_sigmoid(x):
  return x*(1-x)
  return 1/(1 + np.exp(-x))

can easily lead to overflow:

your_sigmoid(20)
# -380

The other issue is that your derivative is wrong; it should be:

def Sigmoid_Derivative(x):
    return sigmoid(x) * (1-sigmoid(x))

See the Derivative of sigmoid function thread at Math.SE, as well as the discussion here.