Neural-network-based zero-sum game for discrete-time nonlinear systems via iterative adaptive dynamic programming algorithm

Authors:
Derong Liu;Hongliang Li;Ding Wang
Affiliations:
State Key Laboratory of Management and Control for Complex Systems, Institute of Automation, Chinese Academy of Sciences, Beijing 100190, PR China;State Key Laboratory of Management and Control for Complex Systems, Institute of Automation, Chinese Academy of Sciences, Beijing 100190, PR China;State Key Laboratory of Management and Control for Complex Systems, Institute of Automation, Chinese Academy of Sciences, Beijing 100190, PR China
Venue:
Neurocomputing
Year:
2013

Citing 31
Cited 0

Galerkin approximations of the generalized Hamilton-Jacobi-Bellman equation

Automatica (Journal of IFAC)
Introduction to Reinforcement Learning

Introduction to Reinforcement Learning
Neuro-Dynamic Programming

Neuro-Dynamic Programming
Handbook of Learning and Approximate Dynamic Programming (IEEE Press Series on Computational Intelligence)

Handbook of Learning and Approximate Dynamic Programming (IEEE Press Series on Computational Intelligence)
Brief paper: Model-free Q-learning designs for linear discrete-time zero-sum games with application to H-infinity control

Automatica (Journal of IFAC)
Brief paper: A neural network solution for fixed-final time optimal control of nonlinear systems

Automatica (Journal of IFAC)
Brief paper: Adaptive optimal control for continuous-time linear systems based on policy iteration

Automatica (Journal of IFAC)
A game theoretic algorithm to compute local stabilizing solutions to HJBI equations in nonlinear H∞ control

Automatica (Journal of IFAC)
2009 Special Issue: Neural network approach to continuous-time direct adaptive optimal control for partially unknown nonlinear systems

Neural Networks
2009 Special Issue: Optimal control of unknown affine nonlinear discrete-time systems using offline-trained neural networks with proof of convergence

Neural Networks
Neural-network-based near-optimal control for a class of discrete-time affine nonlinear systems with control constraints

IEEE Transactions on Neural Networks
Reinforcement learning and adaptive dynamic programming for feedback control

IEEE Circuits and Systems Magazine
Online actor-critic algorithm to solve the continuous-time infinite horizon optimal control problem

Automatica (Journal of IFAC)
Brief paper: Model-free H∞ control design for unknown linear discrete-time systems via Q-learning with LMI

Automatica (Journal of IFAC)
Optimal control laws for time-delay systems with saturating actuators based on heuristic dynamic programming

Neurocomputing
Brief paper: An iterative adaptive dynamic programming method for solving a class of nonlinear zero-sum differential games

Automatica (Journal of IFAC)
Finite-horizon neuro-optimal tracking control for a class of discrete-time nonlinear systems using adaptive dynamic programming approach

Neurocomputing
Adaptive dynamic programming

IEEE Transactions on Systems, Man, and Cybernetics, Part C: Applications and Reviews
Adaptive dynamic programming-based optimal control of unknown nonaffine nonlinear discrete-time systems with proof of convergence

Neurocomputing
Adaptive Critic Designs for Discrete-Time Zero-Sum Games With Application to Control

IEEE Transactions on Systems, Man, and Cybernetics, Part B: Cybernetics
Reinforcement Learning Neural-Network-Based Controller for Nonlinear Discrete-Time Systems With Input Constraints

IEEE Transactions on Systems, Man, and Cybernetics, Part B: Cybernetics
Discrete-Time Nonlinear HJB Solution Using Approximate Dynamic Programming: Convergence Proof

IEEE Transactions on Systems, Man, and Cybernetics, Part B: Cybernetics
A Novel Infinite-Time Optimal Tracking Control Scheme for a Class of Discrete-Time Nonlinear Systems via the Greedy HDP Iteration Algorithm

IEEE Transactions on Systems, Man, and Cybernetics, Part B: Cybernetics
Guest Editorial - Special Issue on Adaptive Dynamic Programming and Reinforcement Learning in Feedback Control

IEEE Transactions on Systems, Man, and Cybernetics, Part B: Cybernetics
Control of Nonaffine Nonlinear Discrete-Time Systems Using Reinforcement-Learning-Based Linearly Parameterized Neural Networks

IEEE Transactions on Systems, Man, and Cybernetics, Part B: Cybernetics
Nearly optimal control laws for nonlinear systems with saturating actuators using a neural network HJB approach

Automatica (Journal of IFAC)
Adaptive critic designs

IEEE Transactions on Neural Networks
Neurodynamic Programming and Zero-Sum Games for Constrained Control Systems

IEEE Transactions on Neural Networks
Reinforcement-Learning-Based Dual-Control Methodology for Complex Nonlinear Discrete-Time Systems With Application to Spark Engine EGR Operation

IEEE Transactions on Neural Networks
Adaptive Dynamic Programming for Finite-Horizon Optimal Control of Discrete-Time Nonlinear Systems With -Error Bound

IEEE Transactions on Neural Networks
Approximate Dynamic Programming for Optimal Stationary Control With Control-Dependent Noise

IEEE Transactions on Neural Networks - Part 2

Quantified Score

Hi-index	0.01

Visualization

Abstract

In this paper, we solve the zero-sum game problems for discrete-time affine nonlinear systems with known dynamics via iterative adaptive dynamic programming algorithm. First, a greedy heuristic dynamic programming iteration algorithm is developed to solve the zero-sum game problems, which can be used to solve the Hamilton-Jacobi-Isaacs equation associated with H"~ optimal regulation control problems. The convergence analysis in terms of value function and control policy is provided. To facilitate the implementation of the algorithm, three neural networks are used to approximate the control policy, the disturbance policy, and the value function, respectively. Then, we extend the algorithm to H"~ optimal tracking control problems through system transformation. Finally, two simulation examples are presented to demonstrate the effectiveness of the proposed scheme.