How expected improvement acquisition function of Bayesian optimization is maximized?

Hi, everyone, do you know what solvers MATLAB used to maximize the acquisition functions (e.g., probability of improvement, expected improvement) of Bayesian optimization. And what methods MATLAB used to maximize the acquisition functions? Or how it find next point to evaluate? Thanks.

Answer 1

[Yoga]

Hi @Jiafeng Ye!

I understand that you would like to know what solvers/methods MATLAB uses to maximize the acquisition functions. 'bayesopt' estimates the smallest feasible mean of the posterior distribution 'μQ' (xbest) by sampling several thousand points within the variable bounds, taking several of the best (low mean value) feasible points, and improving them using local search, to find the ostensible best feasible point.

You can refer to the following links to know more about the methods MATLAB uses for Bayesian optimization:

1. https://www.mathworks.com/help/stats/bayesian-optimization-algorithm.html

I hope this helps resolve resolve your issue.

[Jiafeng Ye]

Good morning, Yoga. Thank you for your answer.

I found the function 'fminsearch' in the codes of 'bayopt' to maximize (minimize the negative) the acquisition functions. Based on this function, I think MATLAB uses simplex search method (https://www.mathworks.com/help/matlab/ref/fminsearch.html#bvadxhn-12) to find the maximal acquistion function value in each iteration.

Thank you again for your answer. Please correct me if I misunderstand the codes of 'bayopt'.

Jiafeng