Why is not possible to put the SpecifyObjectiveGradient option to true in case of Levenberg-Marquardt?

As I know from the theory, the Levenberg-Marquardt algorithm also needs to compute the Gradient and the Hessian. If I set the SpecifyObjectiveGradient property to true and I define the gradients explicitly, the lsqnonlin command chooses the trust-region-reflective method instead of the Levenberg-Marquardt to solve the problem. I don't understand, why I cannot reduce the number of function evaluations for the Levenberg-Marquardt algorithm by defining the gradient? Thanks for your help!

Answer 1

[Elizabeth Reese]

The default algorithm for lsqnonlin is the trust-region-reflective method. In order to use the Levenberg-Marquardt algorithm and the SpecifyObjectiveGradient flag, you need you specify both in the options for the lsqnonlin problem. For example:

options = optimoptions(@lsqnonlin,'Algorithm','levenberg-marquardt','SpecifyObjectiveGradient',true);

Then you can specify the gradient using the input function described at the link below.

https://www.mathworks.com/help/optim/ug/lsqnonlin.html#inputarg_fun

Finally, you can call lsqnonlin as with the following syntax:

x = lsqnonlin(fun,x0,lb,ub,options);

[cinna]

Thank you for your answer. I know how to do the settings for the gradient, my problem is, that Matlab is not computing the way the settings are. Immediately after calling the function you wrote above, asking then to show the settings again, one can see that Matlab actually has chosen the trust-region-reflective algorithm instead of levenvberg-marquardt. But if the SpecifyObjectiveGradient is set to false and the method to Levenberg-Marquardt, Matlab is computing with L.-M. indeed. This is what I do not understand.

[Alan Weiss]

I cannot reproduce your observation. When I run lsqnonlin with Algorithm = 'levenberg-marquardt' and SpecifyObjectiveGradient = true, and use the debugger to follow the code path that lsqnonlin takes, then I see it using the Levenberg-Marquardt algorithm.

Alan Weiss

MATLAB mathematical toolbox documentation