Calculate the error using a sigmoid function in backpropagation

Question

I have a quick question regarding backpropagation. I am looking at the following:

http://www4.rgu.ac.uk/files/chapter3%20-%20bp.pdf

In this paper, it says to

Answer 1

The reason you need this is that you are calculating the derivative of the error function with respect to the neuron's inputs.

When you take the derivative via the chain rule, you need to multiply by the derivative of the neuron's activation function (which happens to be a sigmoid)

Here's the important math.

Calculate the derivative of the error on the neuron's inputs via the chain rule:

E = -(target - output)^2

dE/dinput = dE/doutput * doutput/dinput

Work out doutput/dinput:

output = sigmoid (input)

doutput/dinput = output * (1 - output)    (derivative of sigmoid function)

therefore:

dE/dinput = 2 * (target - output) * output * (1 - output)

Answer 2

The choice of the sigmoid function is by no means arbitrary. Basically you are trying to estimate the conditional probability of a class label given some sample. If you take the absolute value, you are doing something different, and you will get different results.

For a practical introduction in the topic I would recommend you to check out the online Machine Learning course by Prof. Andrew Ng

https://www.coursera.org/course/ml

and the book by Prof. Christopher Bishop for an in depth study on the topic

http://www.amazon.com/Neural-Networks-Pattern-Recognition-Christopher/dp/0198538642/ref=sr_1_1?ie=UTF8&qid=1343123246&sr=8-1&keywords=christopher+bishop+neural+networks