Programming a Basic Neural Network from scratch in MATLAB

Question

I have asked a few questions about neural networks on this website in the past and have gotten great answers, but I am still struggling to implement one for myself. This is quit

Answer 1

Your Neural network seems alright, although the kind of training you're trying to do is quite in-efficient if you're training against labeled data as you're doing. In that case I would suggest looking into Back-propagation

About your error when training: Your error message hints at the problem: dimensions are not consistent

As argument x0 in fminsearch which is the initial guess for the optimizer, you send [W1, W2] but from what I can see, these matrices don't have the same number of rows, and therefore you can't add them together like that. I would suggest modifying your cost-function to take a vector as argument and then form your weight-vectors for different layers from that one vector.

You are also not supplying the cost-function correctly to fminsearch as you are just evaluating cost with w1, w2, Xtrain and Ytrain in-place.

According to the documentation (it's been years since I used Matlab) it seems like you pass the pointer to the cost-function as fminsearch(cost, [W1; W2])

EDIT: You could express your weights and modify your code as follows:

global Xtrain
global Ytrain
W = [W1; W2]
fminsearch(cost, W)

Cost-function must be modified such that it doesn't take Xtrain, Ytrain as input because fminsearch will then try to optimize those too. Modify your cost-function like this:

function [ C ] = cost( W )
   W1 = W[1:2,:]
   W2 = W[3,:]
   global Xtrain
   global Ytrain
   ...