How to convert deep learning gradient descent equation into python

大城市里の小女人 提交于 2019-12-04 21:31:38

After going through the code and notes a few times was finally able to figure out the error.

First it needs calculating Z and then pass it to the sigmoid function, instead of X

Formula for Z = w(T)X+b. So in python this is calculated as below

Z=np.dot(w.T,X)+b

Then calculate A by passing z to sigmoid function

A = sigmoid(Z)

Then dw can be calculated as below

dw=np.dot(X,(A-Y).T)/m

Calculation of the other variables; cost and derivative of b will be as follows

cost = -1*((np.sum((Y*np.log(A))+((1-Y)*(np.log(1-A))),axis=1))/m) 
db = np.sum((A-Y),axis=1)/m

You can calculate A,cost,dw,db as the following:

A = sigmoid(np.dot(w.T,X) + b)     
cost = -1 / m * np.sum(Y*np.log(A)+(1-Y)*np.log(1-A)) 

dw = 1/m * np.dot(X,(A-Y).T)
db = 1/m * np.sum(A-Y)

where sigmoid is :

def sigmoid(z):
    s = 1 / (1 + np.exp(-z))    
    return s
JakeJ
def sigmoid(x):
      #You have it right
      return 1/(1 + np.exp(-x))

def derivSigmoid(x):
      return sigmoid(x) * (1 - sigmoid(x))

error = targetSample - output

#Make sure to keep the sigmoided value around.  For instance, an output that has already been sigmoided can be used to get the sigmoid derivative faster (output = sigmoid(x)):
dOutput = output * (1 - output)

Looks like you're already working on the backprop. Just thought I'd help simplify some of the forward prop for you.

易学教程内所有资源均来自网络或用户发布的内容,如有违反法律规定的内容欢迎反馈
该文章没有解决你所遇到的问题?点击提问,说说你的问题,让更多的人一起探讨吧!