Minimizing linear equation Ax=b using gradient descent

I want to find the error in the solution to Ax=b, using gradient descent.

E=||Ax-b||^2

x = [x1;x2], where x1 and x2 range between -5 and 5, with step size 0.2 for each direction.

How do I use Gradient Descent to search for a local minimum with know step size of 0.2, learning rate= 0.1. The search should stop when the difference between previous and current value is 0.002. I am to find solution for x using Gradient Descent, as well error E.

[Jan]

This sounds like a homework question. Please post, what you have tried so far and ask a specific question. The forum will not solve your homework.

[Tevin]

%I am trying to find the the Error for each pair of X,Y grid value, 

x=(-5:0.2:5);
y=(-5:0.2:5);

[X,Y]= meshgrid(x,y);

A=[2 0;3 1;2 2];
b=[2;3;4];

% E=||Ax-b||^2   where x=[X,Y]
for i=1:length(X)

Error=(norm(A*[X,Y] -b))^2;  %I am getting Matlab error here but I do not know how to correct this.

end 
surf(X,Y,Z)

% After getting the surf plot, I will perform the gradient descent with a
% function I wrote.

[Hiro Yoshino]

You need to derive the derivative of the Error function. Gradient Descent requires it to move the point of interest to the next.

[Tevin]

Thank you. The function that I wrote already does that. My problem is that I struggle to calculate error for all the grid values (X,Y). The array sizes are incompatible but I am not sure how to fix that.

Answer 1

[Matt J]

[X1,X2]= meshgrid(-5:0.2:5);

x=[X1(:)';X2(:)'];

E=vecnorm( A*x-b, 2,1);
  E=reshape(E,size(X1)); %if desired

[Tevin]

Thank you. What does the 2,1 mean in line 3? Also is the same as (A*x-b)^2?

[Torsten]

It's sqrt(sum((A*x-b).^2))

[Tevin]

Thank you both. This really helped