Multi-agent differential graphical games: Online adaptive learning solution for synchronization with optimality

  • Authors:

  • Kyriakos G. Vamvoudakis;Frank L. Lewis;Greg R. Hudas

  • Affiliations:

  • Center for Control, Dynamical-systems and Computation (CCDC), University of California, Santa Barbara, CA 93106-9560, USA;Automation and Robotics Research Institute, The University of Texas at Arlington, 7300 Jack Newell Blvd. S., Ft. Worth, TX 76118, USA;U.S. Army RDECOM-TARDEC, Joint Center for Robotics (JCR), Warren, MI 48397-5000, USA

  • Venue:
  • Automatica (Journal of IFAC)
  • Year:
  • 2012

Quantified Score

Hi-index 22.15

Visualization

Abstract

Multi-agent systems arise in several domains of engineering and can be used to solve problems which are difficult for an individual agent to solve. Strategies for team decision problems, including optimal control, N-player games (H-infinity control and non-zero sum), and so on are normally solved for off-line by solving associated matrix equations such as the coupled Riccati equations or coupled Hamilton-Jacobi equations. However, using that approach players cannot change their objectives online in real time without calling for a completely new off-line solution for the new strategies. Therefore, in this paper we bring together cooperative control, reinforcement learning, and game theory to present a multi-agent formulation for the online solution of team games. The notion of graphical games is developed for dynamical systems, where the dynamics and performance indices for each node depend only on local neighbor information. It is shown that standard definitions for Nash equilibrium are not sufficient for graphical games and a new definition of ''Interactive Nash Equilibrium'' is given. We give a cooperative policy iteration algorithm for graphical games that converges to the best response when the neighbors of each agent do not update their policies, and to the cooperative Nash equilibrium when all agents update their policies simultaneously. This is used to develop methods for online adaptive learning solutions of graphical games in real time along with proofs of stability and convergence.