fminunc giving correct optimized values for anonymously defined function, BUT wrong value for .m function

I have a function at first defined anonymously inside the same script. After performing optimization through fminunc, it provides acceptable optimized parameter values.

f=@(A)0;

for i=1:1:size(c,1)

disp(i)

v(i,1)=(p2(i,1)-p1(i,1))/(sqrt(((p2(i,1)-p1(i,1)).^2)+((p2(i,2)-p1(i,2)).^2)+((p2(i,3)-p1(i,3)).^2)));

v(i,2)=(p2(i,2)-p1(i,2))/(sqrt(((p2(i,1)-p1(i,1)).^2)+((p2(i,2)-p1(i,2)).^2)+((p2(i,3)-p1(i,3)).^2)));

v(i,3)=(p2(i,3)-p1(i,3))/(sqrt(((p2(i,1)-p1(i,1)).^2)+((p2(i,2)-p1(i,2)).^2)+((p2(i,3)-p1(i,3)).^2)));

x=c(i,24)*((n-k+2)/127.59);

N11=BSpline1(k,t,x,n,periodic);

N22=BSpline2(k,t,x,N11,periodic);

N33=BSpline3(k,t,x,N22,periodic);

fx=@(A)0;fy=@(A)0;fz=@(A)0;

for j=1:1:(n+1)

fx=@(A)(fx(A)+A(j,1)*N33(j,1));

fy=@(A)(fy(A)+A(j,2)*N33(j,1));

fz=@(A)(fz(A)+A(j,3)*N33(j,1));

end

f=@(A)(f(A)+((p1(i,1)-fx(A))^2+(p1(i,2)-fy(A))^2+(p1(i,3)-fz(A))^2-(((p1(i,1)-fx(A))*v(i,1))+((p1(i,2)-fy(A))*v(i,2))+((p1(i,3)-fz(A))*v(i,3)))^2));

end

A0=ones(n+1,3);

[Aopt,fopt]=fminunc(f,A0);

But this implementation is very slow and to make it run faster I have separatly defined the function and also instead of for loops I used vector and matrix definition directly.

for i=1:1:size(c,1)

disp(i)

v(i,1)=(p2(i,1)-p1(i,1))/(sqrt(((p2(i,1)-p1(i,1)).^2)+((p2(i,2)-p1(i,2)).^2)+((p2(i,3)-p1(i,3)).^2)));

v(i,2)=(p2(i,2)-p1(i,2))/(sqrt(((p2(i,1)-p1(i,1)).^2)+((p2(i,2)-p1(i,2)).^2)+((p2(i,3)-p1(i,3)).^2)));

v(i,3)=(p2(i,3)-p1(i,3))/(sqrt(((p2(i,1)-p1(i,1)).^2)+((p2(i,2)-p1(i,2)).^2)+((p2(i,3)-p1(i,3)).^2)));

x=c(i,24)*((n-k+2)/128.689);

N11(:,i)=BSpline1(k,t,x,n,periodic);

N22(:,i)=BSpline2(k,t,x,N11,periodic);

N33(:,i)=BSpline3(k,t,x,N22,periodic);

end

% function

function f=myObjective(A,N33,p1,v)

fxyz=N33.'*A;

D=p1-fxyz;

E1=D.^2;

E2=E1';

E3=sum(E2);

F1=(D.*v);

F2=F1';

F3=sum(F2);

F4=F3.^2;

G=E3-F4;

f=sum(G);

end

%%%%%%%

A0=ones(n+1,3);

[Aopt,fopt]=fminunc(@(A)myObjective(A,N33,p1,v),A0);

At same initial values in two cases (anonymous function and .m function) the objective function gives same values which means the function definition in two cases are same.

But after performing optimization in last case (.m function case), I get totally wrong values of input matrix.

n=3 % anonymous function case

Aopt=

-1.53685543885162 6.36319472017170 0.0251668191421513

13.1109550026546 -54.9314786364682 -0.248200752927807

37.2709897604245 49.3077792319788 -0.169381729039393

50.0220902870506 2.38624594683487 0.217368669572959

n=3 % .m function case

Aopt=

0.599435929547115 -26.1229467554614 -0.286717780987037

26.0699535504464 -0.642922198673024 -0.0939110197500202

1.00105690382098 1.00025671501624 0.999976771178033

1 1 1

How do I correct it?

Answer 1

[Matt J]

At same initial values in two cases (anonymous function and .m function) the objective function gives same values which means the function definition in two cases are same.

To be certain the definitions are the same, you probably need to compare the objective function implementations at more than just one A. Also, have you ruled out that both solutions may be valid? How different are the objective function values at the two solutions?

Finally, if the second implementation is correct, then fminunc is unnecessary. There is an analytical solution to your problem, which you could also compare with the other two implementations.

w=v.^2;
g=N33*p1- N33*(w.*p1);
C=N33*N33.';

for i=1:3

    H = C - N33*(w(:,i)*N33.');
        
    A(:,i)=H\g(:,i);
    
end

[Satyajit Ghosh]

If I display the iterations, for anonymous function case I am getting this

Iteration Func-count f(x) Step-size optimality

0 19 1.36337e+07 1.48e+05

1 57 9.77589e+06 6.77861e-05 1.15e+05

2 76 3.1179e+06 1 1.23e+04

3 95 2.99818e+06 1 7.62e+03

4 114 2.93262e+06 1 2.34e+03

5 133 2.92393e+06 1 1.7e+03

6 152 2.9168e+06 1 1.11e+03

7 171 2.91586e+06 1 671

8 190 2.9152e+06 1 409

9 209 2.91492e+06 1 346

10 228 2.9147e+06 1 301

11 247 2.91456e+06 1 324

12 266 2.91445e+06 1 219

13 285 2.9144e+06 1 106

14 304 2.91438e+06 1 75.4

15 323 2.91437e+06 1 56.7

16 342 2.91437e+06 1 33.8

17 361 2.91437e+06 1 22.4

18 380 2.91437e+06 1 15.9

19 399 2.91437e+06 1 9.16

First-order

Iteration Func-count f(x) Step-size optimality

20 418 2.91437e+06 1 4.44

21 437 2.91437e+06 1 2.56

22 456 2.91437e+06 1 2.5

23 475 2.91437e+06 1 1.97

24 494 2.91437e+06 1 1.22

25 513 2.91437e+06 1 2.34

26 532 2.91437e+06 1 1.03

And for .m function I am getting this

Iteration Func-count f(x) Step-size optimality

0 19 1.36337e+07 1.3e+06

1 38 1.16916e+07 7.66998e-07 1.16e+06

2 57 4.62685e+06 1 4.43e+04

3 95 4.47799e+06 10 8.2e+04

4 114 4.31836e+06 1 9.58e+04

5 133 3.91068e+06 1 1.01e+05

6 152 3.79107e+06 1 4.84e+04

7 171 3.77141e+06 1 6.75e+03

8 190 3.77072e+06 1 2.59e+03

9 209 3.77062e+06 1 2.18e+03

10 228 3.7703e+06 1 2.44e+03

11 247 3.76977e+06 1 3.55e+03

12 266 3.76906e+06 1 3.52e+03

13 285 3.76863e+06 1 1.87e+03

14 304 3.76853e+06 1 551

15 323 3.76852e+06 1 423

16 342 3.76851e+06 1 429

17 361 3.7685e+06 1 429

18 380 3.76848e+06 1 377

19 399 3.76845e+06 1 414

First-order

Iteration Func-count f(x) Step-size optimality

20 418 3.76843e+06 1 239

21 437 3.76842e+06 1 62.8

22 456 3.76842e+06 1 8.31

23 475 3.76842e+06 1 1.19

Could you guess something from this?

[Satyajit Ghosh]

Original objective function contain two extra terms which will make it nonseparable. So I am not thinking in that direction.

[Matt J]

But for the moment the objective is separable, so you can compare the analytical result to the two iterative results you've shown above.

[Satyajit Ghosh]

Sorry! After I checked properly, I found that two implementation are not equivalent. These are some values of two functions at various A matrices.

.m function Anonymous function

36231258.356491 39156792.515199

39225691.846561 39413498.295238

38708505.011888 39947618.013650

38711397.246001 39412672.483283

41245274.325280 39099619.819368

39044211.256773 39552230.937709

39826568.903608 39600681.258399

41745908.334861 39283223.331794

40419353.228199 39951881.566275

39374480.958563 39736514.229297

37909505.930636 38829415.026319

38869566.144146 39658157.553641

41179311.037528 38966749.459882

40062381.933204 39477309.992161

40044091.845803 39238422.148523

40352935.907771 39412283.634430

37843711.188815 38913223.366082

38911091.704537 39290172.416156

39161888.494808 39299591.369524

38974170.056678 39380423.107997

[Satyajit Ghosh]

Hi, thank you for your help. I have fixed the issue. Now two implementations are equivalent. Thank you very much.