How does bayesopt() optimise using categorical optimizable variables?

I have been using the bayesopt() function to perform Bayesian optimisation for material design. I have one set of optimisable variables that are of categorical data type. I was wondering how Gaussian process regression is performed when categorical input variables are used? Is it simply a case of using one-hot encoding?

Many thanks,

James

Answer 1

[Don Mathis]

The bayesopt function uses a special technique to handle categorical variables. One-hot coding is not used. Instead, bayesopt encodes the categorical variable as an integer variable, and uses an ARD Gaussian Process kernel with a fixed spatial scale on that dimension that is so small that neighboring integer values have virtually no effect on each other. The value of the objective function model at x=7 has no effect on the model's value at x=8. The result is that the distinct integer values must be probed individually to learn what the objective function is at that value. One-hot coding would probably produce similar behavior but would increase the number of variables and require a constraint between them to make sure only one dimension is probed at a time.

[James Finley]

Dear Don,

Many thanks for your swift and concise response. Do you know of any relevant literature that discusses this encoding method in more detail?

Many thanks,

James

[Don Mathis]

Unfortunately, no. As far as I know this is a novel technique. I know of at least one other B.O. package that encodes categoricals as integers, but then it treats the integer variable like any other, estimating a kernel scale that ties neighboring values together, which to me seems inappropriate.

[James Finley]

Okay Don, thanks for your help.

And I agree that this other way of encoding categoricals sounds incorrect.