1

So i am trying to implement/solve the first programming excersise from Andrew ng`s machine learn cours on coursera. I have trouble implementing linear gradient descent(for one variable) in octave. I don't get the same paramters values back like in the solution but my parameters goes in the same direction(at least i think so). So i may have somewhere in my code a bug. Maybe someone who has more experience than me can enlighten me.

function [theta, J_history] = gradientDescent(X, y, theta, alpha, num_iters)
%GRADIENTDESCENT Performs gradient descent to learn theta
%   theta = GRADIENTDESCENT(X, y, theta, alpha, num_iters) updates theta by 
%   taking num_iters gradient steps with learning rate alpha

% Initialize some useful values
m = length(y); % number of training examples
J_history = zeros(num_iters, 1);

theta1 = theta(1);
theta2 = theta(2);

temp0 = 0;
temp1 = 0;

h = X * theta;
for iter = 1:(num_iters)

    % ====================== YOUR CODE HERE ======================
    % Instructions: Perform a single gradient step on the parameter vector
    %               theta. 
    %
    % Hint: While debugging, it can be useful to print out the values
    %       of the cost function (computeCost) and gradient here.
    %
    temp0 = 0;
    temp1 = 0;
    for i=1:m
        error = (h(i) - y(i));
        temp0 = temp0 + error * X(i, 1));;
        temp1 = temp1 + error * X(i, 2));
    end
    theta1 = theta1 - ((alpha/m) * temp0);
    theta2 = theta2 - ((alpha/m) * temp1);
    theta = [theta1;theta2];

    % ============================================================

    % Save the cost J in every iteration    
    J_history(iter) = computeCost(X, y, theta);

end
end

My exspected results for excersise 1 with theta initialized with [0;0] should be for theta1: -3.6303 and for theta2: 1.1664

But i become as output theta1 is 0.095420 and thetha2 is 0.51890

here's a photo where you can see my function

This is the formula i use for linear gradient descent.

this is my formula for linear gradien descent

where  in the formula this is h(x)

EDIT1: Edited code. Now i got for theta1:

87.587

and for theta2

979.93

  • 1
    In the inner for loop, you are replacing temp0 and temp1 m times, and then just using the last value – Ander Biguri Apr 11 at 14:00
  • thank`s i think this might be the bug. I totally didn't see that i am so stupid. Thank you very much. – Yuto Apr 11 at 14:04
0

I now know what my problem was. I am going to describe it quick for anbody who might be intrested in it. So i accidently calulated the avriable h outside of my loop. So every time in the loop it calulated with the same value.

Here is the fixed code:

function [theta, J_history] = gradientDescent(X, y, theta, alpha, num_iters)
%GRADIENTDESCENT Performs gradient descent to learn theta
%   theta = GRADIENTDESCENT(X, y, theta, alpha, num_iters) updates theta by 
%   taking num_iters gradient steps with learning rate alpha

% Initialize some useful values
m = length(y); % number of training examples
J_history = zeros(num_iters, 1);

theta1 = theta(1);
theta2 = theta(2);

temp0 = 0;
temp1 = 0;
error = 0;

for iter = 1:(num_iters)
    % ====================== YOUR CODE HERE ======================
    % Instructions: Perform a single gradient step on the parameter vector
    %               theta. 
    %
    % Hint: While debugging, it can be useful to print out the values
    %       of the cost function (computeCost) and gradient here.
    %

    h = X * theta; %heres the variable i moved into the loop

    temp0 = 0;
    temp1 = 0;
    for i=1:m
        error = (h(i) - y(i));
        temp0 = temp0 + (error * X(i, 1));
        temp1 = temp1 + (error * X(i, 2));
        %disp(error);
    end
    theta1 = theta1 - ((alpha/m) * temp0);
    theta2 = theta2 - ((alpha/m) * temp1);
    theta = [theta1;theta2];

    % ============================================================

    % Save the cost J in every iteration    
    J_history(iter) = computeCost(X, y, theta);

end
end

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.