DCOB: Action space for reinforcement learning of high DoF robots

Authors:
Akihiko Yamaguchi;Jun Takamatsu;Tsukasa Ogasawara
Affiliations:
Graduate School of Information Science, Nara Institute of Science and Technology, Nara, Japan 630-0192;Graduate School of Information Science, Nara Institute of Science and Technology, Nara, Japan 630-0192;Graduate School of Information Science, Nara Institute of Science and Technology, Nara, Japan 630-0192
Venue:
Autonomous Robots
Year:
2013

Citing 15
Cited 0

The Parti-game Algorithm for Variable Resolution Reinforcement Learning in Multidimensional State-spaces

Machine Learning
Feature-based methods for large scale dynamic programming

Machine Learning - Special issue on reinforcement learning
Multiple paired forward and inverse models for motor control

Neural Networks - Special issue on neural control and robotics: biology and technology
Between MDPs and semi-MDPs: a framework for temporal abstraction in reinforcement learning

Artificial Intelligence
Introduction to Reinforcement Learning

Introduction to Reinforcement Learning
Multiple model-based reinforcement learning

Neural Computation
Q-Cut - Dynamic Discovery of Sub-goals in Reinforcement Learning

ECML '02 Proceedings of the 13th European Conference on Machine Learning
Automatic Discovery of Subgoals in Reinforcement Learning using Diverse Density

ICML '01 Proceedings of the Eighteenth International Conference on Machine Learning
Using Eligibility Traces to Find the Best Memoryless Policy in Partially Observable Markov Decision Processes

ICML '98 Proceedings of the Fifteenth International Conference on Machine Learning
Reinforcement learning with via-point representation

Neural Networks
On-line EM Algorithm for the Normalized Gaussian Network

Neural Computation
Reinforcement learning for a biped robot based on a CPG-actor-critic method

Neural Networks
Learning motor primitives for robotics

ICRA'09 Proceedings of the 2009 IEEE international conference on Robotics and Automation
Algorithms

Algorithms
Universal approximation bounds for superpositions of a sigmoidal function

IEEE Transactions on Information Theory

Quantified Score

Hi-index	0.00

Visualization

Abstract

Reinforcement learning (RL) for robot control is an important technology for future robots since it enables us to design a robot's behavior using the reward function. However, RL for high degree-of-freedom robot control is still an open issue. This paper proposes a discrete action space DCOB which is generated from the basis functions (BFs) given to approximate a value function. The remarkable feature is that, by reducing the number of BFs to enable the robot to learn quickly the value function, the size of DCOB is also reduced, which improves the learning speed. In addition, a method WF-DCOB is proposed to enhance the performance, where wire-fitting is utilized to search for continuous actions around each discrete action of DCOB. We apply the proposed methods to motion learning tasks of a simulated humanoid robot and a real spider robot. The experimental results demonstrate outstanding performance.