Example of 10-fold cross-validation with Neural network classification in MATLAB

Question

I am looking for an example of applying 10-fold cross-validation in neural network.I need something link answer of this question: Example of 10-fold SVM classification in MA

Answer 1

It's a lot simpler to just use MATLAB's crossval function than to do it manually using crossvalind. Since you are just asking how to get the test "score" from cross-validation, as opposed to using it to choose an optimal parameter like for example the number of hidden nodes, your code will be as simple as this:

load fisheriris;

% // Split up species into 3 binary dummy variables
S = unique(species);
O = [];
for s = 1:numel(S)
    O(:,end+1) = strcmp(species, S{s});
end

% // Crossvalidation
vals = crossval(@(XTRAIN, YTRAIN, XTEST, YTEST)fun(XTRAIN, YTRAIN, XTEST, YTEST), meas, O);

All that remains is to write that function fun which takes in input and output training and test sets (all provided to it by the crossval function so you don't need to worry about splitting your data yourself), trains a neural net on the training set, tests it on the test set and then output a score using your preferred metric. So something like this:

function testval = fun(XTRAIN, YTRAIN, XTEST, YTEST)

    net = feedforwardnet(10);
    net = train(net, XTRAIN', YTRAIN');

    yNet = net(XTEST');
    %'// find which output (of the three dummy variables) has the highest probability
    [~,classNet] = max(yNet',[],2);

    %// convert YTEST into a format that can be compared with classNet
    [~,classTest] = find(YTEST);


    %'// Check the success of the classifier
    cp = classperf(classTest, classNet);
    testval = cp.CorrectRate; %// replace this with your preferred metric

end

I don't have the neural network toolbox so I am unable to test this I'm afraid. But it should demonstrate the principle.