Regularized logistic regression code in matlab

Question

I\'m trying my hand at regularized LR, simple with this formulas in matlab:

The cost function:

J(theta) = 1/m*sum((-y_i)*log(h(x_i)-(1-y_i)*log(1-h(x_i))         


        

Answer 1

I used more variables, so you could see clearly what comes from the regular formula, and what comes from "the regularization cost added". Additionally, It is a good practice to use "vectorization" instead of loops in Matlab/Octave. By doing this, you guarantee a more optimized solution.

 function [J, grad] = costFunctionReg(theta, X, y, lambda)

    %Hypotheses
    hx = sigmoid(X * theta);

    %%The cost without regularization
    J_partial = (-y' * log(hx) - (1 - y)' * log(1 - hx)) ./ m;


    %%Regularization Cost Added
    J_regularization = (lambda/(2*m)) * sum(theta(2:end).^2);

    %%Cost when we add regularization
    J = J_partial + J_regularization;

    %Grad without regularization
    grad_partial = (1/m) * (X' * (hx -y));

    %%Grad Cost Added
    grad_regularization = (lambda/m) .* theta(2:end);

    grad_regularization = [0; grad_regularization];

    grad = grad_partial + grad_regularization;