Stochastic optimal control of unknown linear networked control system in the presence of random delays and packet losses

Authors:
Hao Xu;S. Jagannathan;F. L. Lewis
Affiliations:
Electrical and Computer Engineering Department, Missouri University of Science and Technology, Rolla, MO 65401, USA;Electrical and Computer Engineering Department, Missouri University of Science and Technology, Rolla, MO 65401, USA;Electrical Engineering Department, AARI, University of Texas at Arlington, Arlington, TX 76118, USA
Venue:
Automatica (Journal of IFAC)
Year:
2012

Citing 11
Cited 4

Persistence of excitation in linear systems

Systems & Control Letters
Stochastic optimal control: theory and application

Stochastic optimal control: theory and application
Adaptive control: stability, convergence, and robustness

Adaptive control: stability, convergence, and robustness
Stochastic analysis and control of real-time systems with random time delays

Automatica (Journal of IFAC)
Feedback Control of Dynamic Systems

Feedback Control of Dynamic Systems
Stochastic Optimal Control: The Discrete-Time Case

Stochastic Optimal Control: The Discrete-Time Case
Brief paper: Model-free Q-learning designs for linear discrete-time zero-sum games with application to H-infinity control

Automatica (Journal of IFAC)
Optimal control of affine nonlinear discrete-time systems

MED '09 Proceedings of the 2009 17th Mediterranean Conference on Control and Automation
Neural-network-based near-optimal control for a class of discrete-time affine nonlinear systems with control constraints

IEEE Transactions on Neural Networks
Reinforcement Learning and Dynamic Programming Using Function Approximators

Reinforcement Learning and Dynamic Programming Using Function Approximators
Brief Stochastic optimal control and analysis of stability of networked control systems with long delay

Automatica (Journal of IFAC)

Computational adaptive optimal control for continuous-time linear systems with completely unknown dynamics

Automatica (Journal of IFAC)
Integral reinforcement learning and experience replay for adaptive optimal control of partially-unknown constrained-input continuous-time systems

Automatica (Journal of IFAC)
Neural network observer-based networked control for a class of nonlinear systems

Neurocomputing
On hold or drop out-of-order packets in networked control systems

Information Sciences: an International Journal

Quantified Score

Hi-index	22.15

Visualization

Abstract

In this paper, the stochastic optimal control of linear networked control system (NCS) with uncertain system dynamics and in the presence of network imperfections such as random delays and packet losses is derived. The proposed stochastic optimal control method uses an adaptive estimator (AE) and ideas from Q-learning to solve the infinite horizon optimal regulation of unknown NCS with time-varying system matrices. Next, a stochastic suboptimal control scheme which uses AE and Q-learning is introduced for the regulation of unknown linear time-invariant NCS that is derived using certainty equivalence property. Update laws for online tuning the unknown parameters of the AE to obtain the Q-function are derived. Lyapunov theory is used to show that all signals are asymptotically stable (AS) and that the estimated control signals converge to optimal or suboptimal control inputs. Simulation results are included to show the effectiveness of the proposed schemes. The result is an optimal control scheme that operates forward-in-time manner for unknown linear systems in contrast with standard Riccati equation-based schemes which function backward-in-time.