Kalman filter control embedded into the reinforcement learning framework

Authors:
István Szita;András Lőrincz
Affiliations:
Department of Information Systems, Eötvös Loránd University, Pázmány Péter sétány 1/C, H-1117 Budapest, Hungary;Department of Information Systems, Eötvös Loránd University, Pázmány Péter sétány 1/C, H-1117 Budapest, Hungary
Venue:
Neural Computation
Year:
2004

Citing 2
Cited 4

Dynamic model of visual recognition predicts neural response properties in the visual cortex

Neural Computation
Reinforcement Learning Applied to Linear Quadratic Regulation

Advances in Neural Information Processing Systems 5, [NIPS Conference]

Representation and timing in theories of the dopamine system

Neural Computation
Neural network learning of optimal Kalman prediction and control

Neural Networks
Bayesian Reward Filtering

Recent Advances in Reinforcement Learning
Neural learning of Kalman filtering, Kalman control, and system identification

IJCNN'09 Proceedings of the 2009 international joint conference on Neural Networks

Quantified Score

Hi-index	0.00

Visualization

Abstract

There is a growing interest in using Kalman filter models in brain modeling. The question arises whether Kalman filter models can be used on-line not only for estimation but for control. The usual method of optimal control of Kalman filter makes use of off-line backward recursion, which is not satisfactory for this purpose. Here, it is shown that a slight modification of the linear-quadratic-gaussian Kalman filter model allows the on-line estimation of optimal control by using reinforcement learning and overcomes this difficulty. Moreover, the emerging learning rule for value estimation exhibits a Hebbian form, which is weighted by the error of the value estimation.