TK-MARS: An Efficient Approach for Deterministic and Stochastic Black-Box Optimization

Abstract

Surrogate optimization approaches for black-box functions focus on approximating the underlying function, using metamodeling techniques, in order to optimize computationally expensive simulation models. Historically, surrogate optimization models have been validated by deterministic (noiseless) functions with every variable being significant. As a result, many surrogate optimization models used interpolating surrogates. However, many real world experiments often times include parameters that are insignificant and uncertainties associated with the black-box function. Using traditional interpolating surrogate optimization methods can lead to surrogate models with unnecessary predictors and sensitivity to noise. Consequently, a surrogate model with flexible, non-interpolating, and parsimonious characteristics is required to overcome real-world noisy black-box functions with only a subset of important variables. One such surrogate model is, Multivariate Adaptive Regression Splines (MARS) which was initially developed by Friedman. In this study, we propose a modified version of MARS, Tree Knot MARS (TK-MARS), to improve the application of MARS within the surrogate optimization context. TK-MARS is able to identify the peaks and valleys for optimization using a classification and regression tree partitioning method. Furthermore, we develop a smart replication strategy based on hypothesis testing. The Smart-Replication approach identifies the promising points to replicate and the number of replications for each of them.