Reinforcement learning and adaptive dynamic programming for feedback control

Authors:
Frank L. Lewis;Draguna Vrabie
Affiliations:
Automation & Robotics Research Institute, The University of Texas at Arlington;Automation & Robotics Research Institute, The University of Texas at Arlington
Venue:
IEEE Circuits and Systems Magazine
Year:
2009

Citing 41
Cited 16

Universal approximation of an unknown mapping and its derivatives using multilayer feedforward networks

Neural Networks
Neural networks for control

Neural networks for control
Connectionist learning for control: an overview

Neural networks for control
A menu of designs for reinforcement learning over time

Neural networks for control
Computational schemes and neural network models for formulation and control of multijoint arm trajectory

Neural networks for control
Practical Issues in Temporal Difference Learning

Machine Learning
Technical Note: \cal Q-Learning

Machine Learning
The Convergence of TD(λ) for General λ

Machine Learning
Galerkin approximations of the generalized Hamilton-Jacobi-Bellman equation

Automatica (Journal of IFAC)
Introduction to Reinforcement Learning

Introduction to Reinforcement Learning
Oscillations in Neural Systems

Oscillations in Neural Systems
Neuro-Dynamic Programming

Neuro-Dynamic Programming
Learning to Predict by the Methods of Temporal Differences

Machine Learning
Handbook of Learning and Approximate Dynamic Programming (IEEE Press Series on Computational Intelligence)

Handbook of Learning and Approximate Dynamic Programming (IEEE Press Series on Computational Intelligence)
Reinforcement Learning in Continuous Time and Space

Neural Computation
An incremental network for on-line unsupervised classification and topology learning

Neural Networks
Stochastic Learning and Optimization: A Sensitivity-Based Approach (International Series on Discrete Event Dynamic Systems)

Stochastic Learning and Optimization: A Sensitivity-Based Approach (International Series on Discrete Event Dynamic Systems)
A single network adaptive critic (SNAC) architecture for optimal control synthesis for a class of nonlinear systems

Neural Networks
Brief paper: Model-free Q-learning designs for linear discrete-time zero-sum games with application to H-infinity control

Automatica (Journal of IFAC)
Approximate Dynamic Programming: Solving the Curses of Dimensionality (Wiley Series in Probability and Statistics)

Approximate Dynamic Programming: Solving the Curses of Dimensionality (Wiley Series in Probability and Statistics)
Brief paper: Adaptive optimal control for continuous-time linear systems based on policy iteration

Automatica (Journal of IFAC)
2009 Special Issue: Neural network approach to continuous-time direct adaptive optimal control for partially unknown nonlinear systems

Neural Networks
2009 Special Issue: Language and cognition

Neural Networks
Adaptive dynamic programming

IEEE Transactions on Systems, Man, and Cybernetics, Part C: Applications and Reviews
Neural dynamic optimization for control systems. I. Background

IEEE Transactions on Systems, Man, and Cybernetics, Part B: Cybernetics
Neural dynamic optimization for control systems.II. Theory

IEEE Transactions on Systems, Man, and Cybernetics, Part B: Cybernetics
Neural dynamic optimization for control systems.III. Applications

IEEE Transactions on Systems, Man, and Cybernetics, Part B: Cybernetics
Reinforcement Learning Neural-Network-Based Controller for Nonlinear Discrete-Time Systems With Input Constraints

IEEE Transactions on Systems, Man, and Cybernetics, Part B: Cybernetics
Discrete-Time Nonlinear HJB Solution Using Approximate Dynamic Programming: Convergence Proof

IEEE Transactions on Systems, Man, and Cybernetics, Part B: Cybernetics
Issues on Stability of ADP Feedback Controllers for Dynamical Systems

IEEE Transactions on Systems, Man, and Cybernetics, Part B: Cybernetics
Hamilton–Jacobi–Bellman Equations and Approximate Dynamic Programming on Time Scales

IEEE Transactions on Systems, Man, and Cybernetics, Part B: Cybernetics
Guest Editorial - Special Issue on Adaptive Dynamic Programming and Reinforcement Learning in Feedback Control

IEEE Transactions on Systems, Man, and Cybernetics, Part B: Cybernetics
Adaptive-critic based optimal neuro control synthesis for distributed parameter systems

Automatica (Journal of IFAC)
Neuro-controller design for nonlinear fighter aircraft maneuver using fully tuned RBF networks

Automatica (Journal of IFAC)
Nearly optimal control laws for nonlinear systems with saturating actuators using a neural network HJB approach

Automatica (Journal of IFAC)
Adaptive critic designs

IEEE Transactions on Neural Networks
Online learning control by association and reinforcement

IEEE Transactions on Neural Networks
Comparison of heuristic dynamic programming and dual heuristic programming adaptive critics for neurocontrol of a turbogenerator

IEEE Transactions on Neural Networks
Continuous-Time Adaptive Critics

IEEE Transactions on Neural Networks
Robust/Optimal Temperature Profile Control of a High-Speed Aerospace Vehicle Using Neural Networks

IEEE Transactions on Neural Networks
Neural net robot controller with guaranteed tracking performance

IEEE Transactions on Neural Networks

Optimal control laws for time-delay systems with saturating actuators based on heuristic dynamic programming

Neurocomputing
Optimal control for a class of unknown nonlinear systems via the iterative GDHP algorithm

ISNN'11 Proceedings of the 8th international conference on Advances in neural networks - Volume Part II
Multi-player non-zero-sum games: Online adaptive learning solution of coupled Hamilton-Jacobi equations

Automatica (Journal of IFAC)
Finite-horizon neuro-optimal tracking control for a class of discrete-time nonlinear systems using adaptive dynamic programming approach

Neurocomputing
Optimal control of unknown nonaffine nonlinear discrete-time systems based on adaptive dynamic programming

Automatica (Journal of IFAC)
Computational adaptive optimal control for continuous-time linear systems with completely unknown dynamics

Automatica (Journal of IFAC)
Constrained adaptive optimal control using a reinforcement learning agent

Automatica (Journal of IFAC)
Integral Q-learning and explorized policy iteration for adaptive optimal control of continuous-time linear systems

Automatica (Journal of IFAC)
An iterative adaptive dynamic programming algorithm for optimal control of unknown discrete-time nonlinear systems with constrained inputs

Information Sciences: an International Journal
Simultaneous policy update algorithms for learning the solution of linear continuous-time H∞ state feedback control

Information Sciences: an International Journal
Data-based stability analysis of a class of nonlinear discrete-time systems

Information Sciences: an International Journal
Neural-network-based zero-sum game for discrete-time nonlinear systems via iterative adaptive dynamic programming algorithm

Neurocomputing
Neural-network-based optimal tracking control scheme for a class of unknown discrete-time nonlinear systems using iterative ADP algorithm

Neurocomputing
Reinforcement learning algorithms with function approximation: Recent advances and applications

Information Sciences: an International Journal
Integral reinforcement learning and experience replay for adaptive optimal control of partially-unknown constrained-input continuous-time systems

Automatica (Journal of IFAC)
On integral generalized policy iteration for continuous-time linear quadratic regulations

Automatica (Journal of IFAC)

Quantified Score

Hi-index	0.01

Visualization

Abstract

Living organisms learn by acting on their environment, observing the resulting reward stimulus, and adjusting their actions accordingly to improve the reward. This action-based or Reinforcement Learning can capture notions of optimal behavior occurring in natural systems. We describe mathematical formulations for Reinforcement Learning and a practical implementation method known as Adaptive Dynamic Programming. These give us insight into the design of controllers for man-made engineered systems that both learn and exhibit optimal behavior.