Optimize Like a Pro with Bayesian Optimization
Bayesian optimization (BayesOpt) is a powerful tool that has found wide application in global optimization tasks, such as hyperparameter tuning, protein engineering, synthetic chemistry, robot learning, and even baking cookies. One of the main reasons BayesOpt is so useful for these problems is because they all involve optimizing black-box functions that are expensive to evaluate. A black-box function's underlying mapping from inputs (configurations of the thing we want to optimize) to outputs (a measure of performance) is unknown. However, we can attempt to understand its internal workings by evaluating the function for different combinations of inputs. Because each evaluation can be computationally expensive, we need to find the best inputs in as few evaluations as possible. BayesOpt works by repeatedly constructing a surrogate model of the black-box function and strategically evaluating the function at the most promising or informative input location, given the information obser...