2009 Special Issue: Neural network approach to continuous-time direct adaptive optimal control for partially unknown nonlinear systems

Authors:
Draguna Vrabie;Frank Lewis
Affiliations:
Automation and Robotics Research Institute, University of Texas at Arlington, 7300 Jack Newell Blvd. S., Fort Worth, TX 76118, USA;Automation and Robotics Research Institute, University of Texas at Arlington, 7300 Jack Newell Blvd. S., Fort Worth, TX 76118, USA
Venue:
Neural Networks
Year:
2009

Citing 10
Cited 14

Universal approximation of an unknown mapping and its derivatives using multilayer feedforward networks

Neural Networks
Galerkin approximations of the generalized Hamilton-Jacobi-Bellman equation

Automatica (Journal of IFAC)
Oscillations in Neural Systems

Oscillations in Neural Systems
Reinforcement Learning in Continuous Time and Space

Neural Computation
2009 Special Issue: Language and cognition

Neural Networks
2009 Special Issue: Coordinated machine learning and decision support for situation awareness

Neural Networks
2009 Special Issue: Intelligence in the brain: A theory of how it works and how to build it

Neural Networks
Nearly optimal control laws for nonlinear systems with saturating actuators using a neural network HJB approach

Automatica (Journal of IFAC)
Adaptive critic designs

IEEE Transactions on Neural Networks
Continuous-Time Adaptive Critics

IEEE Transactions on Neural Networks

Reinforcement learning and adaptive dynamic programming for feedback control

IEEE Circuits and Systems Magazine
2012 Special Issue: An iterative ε-optimal control scheme for a class of discrete-time nonlinear systems with unfixed initial state

Neural Networks
Optimal control of unknown nonaffine nonlinear discrete-time systems based on adaptive dynamic programming

Automatica (Journal of IFAC)
Computational adaptive optimal control for continuous-time linear systems with completely unknown dynamics

Automatica (Journal of IFAC)
Neural networks letter: Controllability and optimal control of a temporal Boolean network

Neural Networks
An iterative adaptive dynamic programming algorithm for optimal control of unknown discrete-time nonlinear systems with constrained inputs

Information Sciences: an International Journal
Output feedback direct adaptive neural network control for uncertain SISO nonlinear systems using a fuzzy estimator of the control error

Neural Networks
A novel actor-critic-identifier architecture for approximate optimal control of uncertain nonlinear systems

Automatica (Journal of IFAC)
Neural-network-based zero-sum game for discrete-time nonlinear systems via iterative adaptive dynamic programming algorithm

Neurocomputing
Neuro-optimal control for a class of unknown nonlinear dynamic systems using SN-DHP technique

Neurocomputing
Neural-network-based optimal tracking control scheme for a class of unknown discrete-time nonlinear systems using iterative ADP algorithm

Neurocomputing
Reinforcement learning algorithms with function approximation: Recent advances and applications

Information Sciences: an International Journal
Integral reinforcement learning and experience replay for adaptive optimal control of partially-unknown constrained-input continuous-time systems

Automatica (Journal of IFAC)
Dual Heuristic dynamic Programming for nonlinear discrete-time uncertain systems with state delay

Neurocomputing

Quantified Score

Hi-index	0.01

Visualization

Abstract

In this paper we present in a continuous-time framework an online approach to direct adaptive optimal control with infinite horizon cost for nonlinear systems. The algorithm converges online to the optimal control solution without knowledge of the internal system dynamics. Closed-loop dynamic stability is guaranteed throughout. The algorithm is based on a reinforcement learning scheme, namely Policy Iterations, and makes use of neural networks, in an Actor/Critic structure, to parametrically represent the control policy and the performance of the control system. The two neural networks are trained to express the optimal controller and optimal cost function which describes the infinite horizon control performance. Convergence of the algorithm is proven under the realistic assumption that the two neural networks do not provide perfect representations for the nonlinear control and cost functions. The result is a hybrid control structure which involves a continuous-time controller and a supervisory adaptation structure which operates based on data sampled from the plant and from the continuous-time performance dynamics. Such control structure is unlike any standard form of controllers previously seen in the literature. Simulation results, obtained considering two second-order nonlinear systems, are provided.