Technical Note: \cal Q-Learning
Machine Learning
Brief paper: Communication and control co-design for networked control systems
Automatica (Journal of IFAC)
Reinforcement learning: a survey
Journal of Artificial Intelligence Research
| Hi-index | 0.00 |
Networked control system (NCS) has become the focus of many recent researches in the control field, with topics covering from scheduling methods, modeling and compensation schemes for the network-induced delay, to stability analysis and controller design strategies, etc. While the majority of researches are carried out on linear NCS, little has been done so far to investigate nonlinear NCS with more complex NCS architecture for nonlinear plants. The main objective of this paper is to propose two-layer networked learning control system architecture that is suitable for complex plants. In proposed architecture, the local controller communicates with the sensors and actuators that are attached to the plant through the first layer communication network, and the network is typically some kind of field bus dedicated to the real-time control. Through the second layer communication network, local controller also communicates with computer systems that typically functions as learning agent. For such a learning agent, firstly, a packet-discard strategy is developed to counteract network-induced delay, data packets that are out of order, and data packet loss. Then cubic spline interpolator is employed to compensate lost data. Finally, the output of learning agent using Q-learning strategy dynamically is used to tune the control signal of local controller. Simulation results of a nonlinear heating, ventilation and air-conditioning (HVAC) system demonstrate the effectiveness of the proposed architecture.