Control of a free-falling cat by policy-based reinforcement learning

Authors:
Daichi Nakano;Shin-ichi Maeda;Shin Ishii
Affiliations:
Graduate School of Informatics, Kyoto University, Uji, Kyoto, Japan;Graduate School of Informatics, Kyoto University, Uji, Kyoto, Japan;Graduate School of Informatics, Kyoto University, Uji, Kyoto, Japan
Venue:
ICANN'12 Proceedings of the 22nd international conference on Artificial Neural Networks and Machine Learning - Volume Part II
Year:
2012

Citing 2
Cited 0

2008 Special Issue: Reinforcement learning of motor skills with policy gradients

Neural Networks
Infinite-horizon policy-gradient estimation

Journal of Artificial Intelligence Research

Quantified Score

Hi-index	0.00

Visualization

Abstract

Autonomous control of nonholonomic systems is one big challenge, because there is no unified control method that can handle any nonholonomic systems even if the dynamics are known. To this challenge, in this study, we propose a reinforcement learning (RL) approach which enables the controller to acquire an appropriate control policy even without knowing the detailed dynamics. In particular, we focus on the control problem of a free-falling cat system whose dynamics are highly-nonlinear and nonholonomic. To accelerate the learning, we take the policy gradient method that exploits the basic knowledge of the system, and present an appropriate policy representation for the task. It is shown that this RL method achieves remarkably faster learning than that by the existing genetic algorithm-based method.