On the Use of Surrogate Functions for Mixed Variable Optimization of Simulated Systems

This research considers the efficient numerical solution of linearly constrained mixed variable programming (MVP) problems, inwhich the objective function is a black-box stochastic simulation, function evaluations may be computationally expensive, andderivative information is typically not available. MVP problems are those with a mixture of continuous, integer, and categoricalvariables, the latter of which may take on values only from a predefined list and may even be non-numeric. Mixed VariableGeneralized Pattern Search with Ranking and Selection (MGPS-RS) is the only existing, provably convergent algorithm that can beapplied to this class of problems. Present in this algorithm is an optional framework for constructing and managing less expensivesurrogate functions as a means to reduce the number of true function evaluations that are required to find approximate solutions.In this research, the NOMADm software package, an implementation of pattern search for deterministic MVP problems, is modifiedto incorporate a sequential selection with memory (SSM) ranking and selection procedure for handling stochastic problems. In doingso, the underlying algorithm is modified to make the application of surrogates more efficient. A second class of surrogates based onthe Nadaraya-Watson kernel regression estimator is also added to the software. Preliminary computational testing of the modifiedsoftware is performed to characterize the relative efficiency of selected surrogate functions for mixed variable optimization insimulated systems.