A Continuous Internal-State Controller for Partially Observable Markov Decision Processes

Authors:
Yuki Taniguchi;Takeshi Mori;Shin Ishii
Affiliations:
Graduate School of Informatics, Kyoto University, Gokasho, Uji, Japan 611-0011;Graduate School of Informatics, Kyoto University, Gokasho, Uji, Japan 611-0011;Graduate School of Informatics, Kyoto University, Gokasho, Uji, Japan 611-0011
Venue:
ICANN '08 Proceedings of the 18th international conference on Artificial Neural Networks, Part I
Year:
2008

Citing 7
Cited 0

Simple Statistical Gradient-Following Algorithms for Connectionist Reinforcement Learning

Machine Learning
Introduction to Reinforcement Learning

Introduction to Reinforcement Learning
Multiagent Systems: A Survey from a Machine Learning Perspective

Autonomous Robots
Scalable Internal-State Policy-Gradient Methods for POMDPs

ICML '02 Proceedings of the Nineteenth International Conference on Machine Learning
Value-function approximations for partially observable Markov decision processes

Journal of Artificial Intelligence Research
Planning and acting in partially observable stochastic domains

Artificial Intelligence
Reinforcement learning for cooperative actions in a partially observable multi-agent system

ICANN'07 Proceedings of the 17th international conference on Artificial neural networks

Quantified Score

Hi-index	0.00

Visualization

Abstract

In this study, in order to control partially observable Markov decision processes, we propose a novel framework called continuous state controller (CSC). The CSC incorporates an auxiliary "continuous" state variable, called an internal state, whose stochastic process is Markov. The parameters of the transition probability of the internal state are adjusted properly by a policy gradient-based reinforcement learning, by which the dynamics of the underlying unknown system can be extracted. Computer simulations show that good control of partially observable linear dynamical systems is achieved by our CSC.