Optimization with a vectorized objective function

Hello,

The objective function that I am trying to optimize is highly vectorized meaning that if someone passed thousands of points to it at once, the function would do its job much more efficiently. The problem is that such optimization routines as fminunc are only capable of evaluating one point at a time; at least I could not find a way to turn on this feature if it does exist. I would be very grateful for any suggestions. Thank you.

Best wishes, Ivan

[José-Luis]

What do you mean by a highly vectorized objective function? An objective function returns a scalar. Are you talking about multiple objective optimization? If that is the case, I don't think you can optimize directly. You either need to express it as a scalar or do some sort of Pareto optimization.

[Ivan]

Thank you for your quick reply. By 'a highly vectorized objective function' I meant the following. It is much more efficient to compute the fitness values of many points at once than to compute these values one by one. For example, assume the objective is x^2. Then, what fminunc does is it passes { x1, x2, x3, ... } sequentially, but it would be much better if it passed several points at a time (if the algorithm inside allows), e.g., (x1, x2, x3), (x4, x5, x6), etc. The objective is indeed scalar, but I would like to compute this scalar for many points at once and return a vector.

Answer 1

[Matt J]

What you're talking about might be possible with GA in the Global Optimization Toolbox. Since global optimization algorithms work with a population of many initial points, it makes sense for it to allow you to vectorize these evaluations.

It doesn't really make sense for the purposes of FMINUNC, though. The whole aim in local optimization code like FMINUNC is to work with one initial point (presumed to be a good one) and to keep to a minimum the number of function evaluations that you have to do. Global optimization algs only work with multiple initial points because they have to.

You could, however, use the efficiency you're talking about to help select a good initial point, which would help improve the speed of FMINUNC. Taking your example of

 f=@(x) x.^2

you could do

 xspace=linspace(-2,2,1000);
 [~,i]=min(f(xspace));
 x0=f(xspace(i));

 fminunc(f,x0,...)

[Ivan]

Well, I expected to hear something disappointing like this. I am interested in fminunc since it provides with a Hessian matrix out of the box, which I do need. I can probably compute this matrix after GA, but I somehow tend not to trust GA. It is strange, fminunc seems to be cursed in some sense as it also does not support parallelization with the UseParallel option. Anyway, thank you for your clarification.

[Sean de Wolski]

@Ivan the 'UseParallel' option is used just calculate the central differences for the gradients. If you want to use this, try fmincon() ( Interior-point algorithm is recommended) with bounds and the 'UseParallel' flag on.

[Matt J]

There's also nothing stopping you from using PARFOR and other PCT functions to parallelize operations inside your objective function.

Answer 2

[Shashank Prasanna]

Ivan, MultiStart may just be what the doctor ordered. It kicks off at several different points and takes different paths in an attempt to find a global minima:

http://www.mathworks.com/help/gads/how-globalsearch-and-multistart-work.html