Adaptive Dynamic Programming For High-dimensional, Multicollinear State Spaces

Abstract

Dynamic programming (DP) is a mathematical programming method for optimizing a system changing over time and has been used to solve multi-stage optimization problems in manufacturing systems, environmental engineering, and many other fields. Exact solutions are only possible for small problems or under very limiting restrictions, but computationally practical approximate DP methods now exist. Most continuous-state problems require discretization of the state space. A design and analysis of computer experiments (DACE) approach for approximate DP uses experimental design and statistical modeling to approximate the value function in continuous-state problems. However, ideal experimental designs are orthogonal, and when the state variables are correlated, ideal experimental designs will not appropriately represent the state space. In this dissertation, the Atlanta ozone pollution problem, which is known for having a multicollinear state space, is selected as our case study. For complex applications like air quality, the state transitions are not given as closed form equations. Rather, an advanced photochemical air quality, such as the Atlanta Urban Airshed Model (UAM), can represent state transitions. However, the UAM is computationally impractical to be used directly in DP. Therefore, in adaptive DP (ADP), statistical metamodels are developed to provide computationally practical surrogates for state transitions. In the dissertation, three types of state transition metamodels for the Atlanta UAM are developed and implemented in ADP. The first type ignores the inherent collinearity between ozone concentrations at different times and monitoring sites and constructs metamodels that have deliberately high variance inflation factors (VIFs). The second type addresses the multicollinearity using classical regression analysis techniques to yield low VIFs. Finally, the third type develops metamodels that orthogonalize the state space. Results are compared under the base case of the Atlanta case study and 50 random hypothetical scenarios.