Emergence of multi-step discrete state transition through reinforcement learning with a recurrent neural network

Authors:
Mohamad Faizal Samsudin;Yoshito Sawatsubashi;Katsunari Shibata
Affiliations:
Department of Electrical and Electronic Engineering, Oita University, Oita, Japan,School of Mechatronic, Universiti Malaysia Perlis, Arau, Perlis, Malaysia;Department of Electrical and Electronic Engineering, Oita University, Oita, Japan;Department of Electrical and Electronic Engineering, Oita University, Oita, Japan
Venue:
ICONIP'12 Proceedings of the 19th international conference on Neural Information Processing - Volume Part II
Year:
2012

Citing 3
Cited 0

Learning internal representations by error propagation

Parallel distributed processing: explorations in the microstructure of cognition, vol. 1
Self-organization of distributedly represented multiple behavior schemata in a mirror system: reviews of robot experiments using RNNPB

Neural Networks - 2004 Special issue: New developments in self-organizing systems
Contextual Behaviors and Internal Representations Acquired by Reinforcement Learning with a Recurrent Neural Network in a Continuous State and Action Space Task

Advances in Neuro-Information Processing

Quantified Score

Hi-index	0.00

Visualization

Abstract

For developing a robot that learns long and complicated action sequences in the real-world, autonomous learning of multi-step discrete state transition is significant. To realize the multi-step discrete state transition in a neural network is generally thought to be difficult because of basically the needs to hold the state while performing the transition between the states when needed. In this paper, only through the reinforcement learning using rewards and punishments in a simple learning system consisting of a recurrent neural network (RNN), it is shown that a multi-step discrete state transition emerged through learning in a continuous state-action space. It is shown that in a two-switch task, two states transition represented by two types of hidden nodes emerged through the learning. In addition, it is shown that the contribution of the dynamics by the interaction between the RNN and the environment based on the discrete state transitions leads to repetition of the interesting behavior when no reward is given at the goal.