Paraboloid minimum point always becomes (0,0)

Hi, I'm trying to do linear regression, by minimization of the squared errors. I.e. by minimizing the following pseudo-code

 sum((y_n-(a*x_n+b)

Where n is integers from 1 to the number of data points used for regression. And don't worry, that is not a part of my script, I just don't know how to make math symbols in here :-). Anyhow, I tried an example with 4 points, and ended up with an equation for minimization. I then tried a range of a and b, and compare to the automatic fit values, which are a=0.6 and b=1.75. The code I used for this is this:

a = -1.5:0.1:2.5;
b = -4:0.1:4;

[A,B] = meshgrid(a,b);

Z = 30*(A.^2) + 4*(B.^2) - 20*(A.*B) + (2.25^2 + 3.25^2 + 3.25^2 + 4.25^2);

figure(1);
surf(A,B,Z);
xlabel('a');
ylabel('b');

My problem is that no matter what I do, the minimum of the function is at (a,b) = (0,0) . Now what the heck am I doing wrong here?

Thanks in advance, and have a nice summer.

Answer 1

[Roger Stafford]

Your problem is with the expression

Z = 30*(A.^2) + 4*(B.^2) - 20*(A.*B) + (2.25^2 + 3.25^2 + 3.25^2 + 4.25^2);

No matter what four points you used for your x_n and y_n, there is no way you can obtain this expression as the one to be minimized using least-sum-of-squares methods when comparing with an a = 0.6 and b = 1.75 version. You had better check your arithmetic. In any case please show us the work you did to obtain this Z and I predict we can punch holes in it.

Answer 2

[Jakob Sørensen]

x1 = 1 | y1 = 2.25

x2 = 2 | y2 = 3.25

x3 = 3 | y3 = 3.25

x4 = 4 | y4 = 4.25

x = [x1 x2 x3 x4];
y = [y1 y2 y3 y4];

Those where the data points used... Next up minimize:

sum(y-a*x+b)

Which equals

(y1-a*x1+b)^2 + (y2-a*x2+b)^2 + (y3-a*x3+b)^2 + (y4-a*x4+b)^2

When when I add together, I get the initial term. Which, apparently, is incorrect. Question is, where did I got wrong :-)

[Roger Stafford]

You are trying to expand z = sum((y-(a*x+b))^2), but you have made some errors in algebra:

 z = sum((y-a*x-b)^2) = 
 sum(a^2*x.^2+2*a*b*x+b^2-2*a*x.*y-2*b*y+y.^2) =
 sum(x.^2)*a^2+2*sum(x)*a*b+4*b^2-2*sum(x.*y)*a-2*sum(y)*b+sum(y.^2) =
 30*a^2 + 20*a*b + 4*b^2 - 71*a - 26*b + 44.25

The sum of the products x.*y and the sum of the y's are not zero, so you need to include the two terms -71*a and -26*b. You also have an incorrect sign on the term 20*a*b; it should be a plus sign. If you include these corrections in your meshgrid minimizing procedure, you will get a non-zero answer for a and b. The precise minimum value would actually occur at a = 0.6 and b = 1.75, just as you obtained in your "automatic fit", but these will not show up in your approximating meshgrid values. You can also derive these precise values by setting the two partial derivatives, dz/da and dz/db equal to zero and solving the resulting simultaneous equations.