Optimize fminsearch behaviour for "very good" initial guess

I have a script that calculates the shortest (optical) path between a startpoint and a set of target points (given a set of arbitrary optically active surfaces between the points). This is done through an iterative process using the fminsearch function (with two optimization veriables, azimuth and elevation angles for the ray direction), and it works very well for the first iteration.

However, for the second iteration I use the azimuth and elevation angles found for the previous ray, which gives a very good initial guess. The fminsearch function still works, but it seems to start out behaving somewhat eratically and in a non-optimal manner. Basically, it seems it starts out taking "very large" initial steps, moving far away from the starting guess, and then optimizing from that point. This leads to the ~20 first iterations of fminsearch to be considerably worse than the starting guess.

I have attached three plots for the first, second, and third optimization iterations. As is seen the first iteration is very effective, while the second and third starts out with a large portion of "unnecessary" iterations (@optimplotfval doesn't seem to plot higher function values than the previous achieved minimum value, but all the intial plateau is the behaviour I've previously described)

Given that I have around 1 million or so points that I'm doing this for it would save a considerable amount of time to elimiate this behaviour. Is this something that anyone has any ideas on, or is it just something I have to live with using fminsearch?

[Rik]

Not that it makes your problem less, but for the further iterations the minimizer already starts out pretty small: 3 orders of magnitude smaller than your first cycle.

Of course there should be a better solution, but have you checked if you get there faster if you deliberately edit your initial guess to be a bit off?

[Oscar Frick]

I have tried, but my tests so far gives no improvement in comparison to the very close guess. Also, achieving an "intelligent" worsened first guess for my simulation iterations seems difficult, and a lot of the time I end up increasing the number of fminsearch iterations needed for several of the simulation iterations due to the initial guess being to bad.

Answer 1

[Alan Weiss]

You can read about the fminsearch algorithm here. The algorithm has no way to know that you started at a good point, and makes a bunch of steps to look around to see what is going on. Sorry, that's the algorithm.

For a faster algorithm, I suggest the Optimization Toolbox™ solvers fminunc or fmincon. It is possible that you would have to tweak their options to take slightly larger-than-default finite differences; see Optimizing a Simulation or ODE. Or, if you have Global Optimization Toolbox, you can try patternsearch, which is easy to tweak (set a small initial mesh for an accurate point).

Good luck,

Alan Weiss

MATLAB mathematical toolbox documentation