Distributed Optimal Policies for Multi-Agent Systems under Uncertainties

Abstract

Multi-agent systems (MAS) have attracted increasing attention in the past years due to their wide applications in mobile robots, sensor networks, autonomous driving systems, etc. Along with this trend, developing distributed optimal policies for MAS under uncertainties has become indispensable. Multi-dimensional uncertainties often modulate system dynamics in a complicated fashion, which leads to computational challenges for real-time control. In many practical MAS, each agent also has its own interest to optimize beyond a global objective. Developing distributed optimal control for agents with self-interests is needed. To address the above challenges, this dissertation contributes in two major directions for MAS: 1) computationally-effective real-time optimal policies under multi-dimensional uncertainties, and 2) distributed optimal policies in networked MAS, including graphical games.

In the first direction, we develop a framework to solve optimal control problems for MAS that involve multi-dimensional uncertainties. Two types of uncertain systems are investigated: 1) MAS subject to uncertain agent intentions, and 2) MAS operating in uncertain environments. For the first type, we use stochastic switching models to capture the uncertain intentions and develop an online optimal control solution that integrates an effective uncertainty sampling method called multivariate probabilistic collocation method (MPCM), reinforcement learning, and random state estimation. For the second type, we formulate and investigate two new stochastic differential games, where the system parameters are modulated by multi-dimensional uncertainties. Effective online learning algorithms are designed to solve these games with computational efficiency.

In the second direction, we study the distributed control, Nash optimality, and robustness properties of networked MAS. We point out that in existing graphical game formulations, being global Nash and being distributed are two contradicting properties. We then propose a new graphical game formulation, which promises the existence of solutions that are both distributed and Nash. In addition, we develop a new Lyapunov-based analytical framework for the robustness of networked MAS measured by gain and phase margins. The effect of communication graph topology on the stability margins is analyzed.

Beyond the theoretical contributions, we also apply the developed solutions to diverse practical applications, including air-to-air unmanned aerial vehicle (UAV) communications, UAV traffic management (UTM), and damage pattern estimations in composite materials.