Reinforcement-learning-based output-feedback control of nonstrict nonlinear discrete-time systems with application to engine emission control

Authors:
Peter Shih;Brian C. Kaul;Sarangapani Jagannathan;James A. Drallmeier
Affiliations:
Department of Electrical and Computer Engineering, Missouri University of Science and Technology, Rolla, MO;Department of Mechanical and Aerospace Engineering, Missouri University of Science and Technology, Rolla, MO;Department of Electrical and Computer Engineering, Missouri University of Science and Technology, Rolla, MO;Department of Mechanical and Aerospace Engineering, Missouri University of Science and Technology, Rolla, MO
Venue:
IEEE Transactions on Systems, Man, and Cybernetics, Part B: Cybernetics
Year:
2009

Citing 10
Cited 0

Nonlinear and Adaptive Control Design

Nonlinear and Adaptive Control Design
Neural Network Control of Robot Manipulators and Nonlinear Systems

Neural Network Control of Robot Manipulators and Nonlinear Systems
Neuro-Dynamic Programming

Neuro-Dynamic Programming
Handbook of Learning and Approximate Dynamic Programming (IEEE Press Series on Computational Intelligence)

Handbook of Learning and Approximate Dynamic Programming (IEEE Press Series on Computational Intelligence)
Adaptive dynamic programming

IEEE Transactions on Systems, Man, and Cybernetics, Part C: Applications and Reviews
Reinforcement learning-based output feedback control of nonlinear systems with input constraints

IEEE Transactions on Systems, Man, and Cybernetics, Part B: Cybernetics
Online learning control by association and reinforcement

IEEE Transactions on Neural Networks
Adaptive output feedback control of uncertain nonlinear systems using single-hidden-layer neural networks

IEEE Transactions on Neural Networks
Neural Network Controller Development and Implementation for Spark Ignition Engines With High EGR Levels

IEEE Transactions on Neural Networks
Stochastic choice of basis functions in adaptive function approximation and the functional-link net

IEEE Transactions on Neural Networks

Quantified Score

Hi-index	0.00

Visualization

Abstract

A novel reinforcement-learning-based output adaptive neural network (NN) controller, which is also referred to as the adaptive-critic NN controller, is developed to deliver the desired tracking performance for a class of nonlinear discrete-time systems expressed in nonstrict feedback form in the presence of bounded and unknown disturbances. The adaptive-critic NN controller consists of an observer, a critic, and two action NNs. The observer estimates the states and output, and the two action NNs provide virtual and actual control inputs to the nonlinear discrete-time system. The critic approximates a certain strategic utility function, and the action NNs minimize the strategic utility function and control inputs. All NN weights adapt online toward minimization of a performance index, utilizing the gradientdescent-based rule, in contrast with iteration-based adaptive-critic schemes. Lyapunov functions are used to show the stability of the closed-loop tracking error, weights, and observer estimates. Separation and certainty equivalence principles, persistency of excitation condition, and linearity in the unknown parameter assumption are not needed. Experimental results on a spark ignition (SI) engine operating lean at an equivalence ratio of 0.75 show a significant (25%) reduction in cyclic dispersion in heat release with control, while the average fuel input changes by less than 1% compared with the uncontrolled case. Consequently, oxides of nitrogen (NOx) drop by 30%, and unburned hydrocarbons drop by 16% with control. Overall, NOx's are reduced by over 80% compared with stoichiometric levels.