Learning to pour with a robot arm combining goal and shape learning for dynamic movement primitives

Authors:
Minija Tamosiunaite;Bojan Nemec;Aleš Ude;Florentin WöRgöTter
Affiliations:
University Göttingen, Institute for Physics 3 - Biophysics, Bernstein Center for Computational Neuroscience, Friedrich-Hund-Platz 1, 37077 Göttingen, Germany and Vytautas Magnus Universi ...;Joef Stefan Institute, Department of Automatics, Biocybernetics, and Robotics, Jamova 39, 1000 Ljubljana, Slovenia;Joef Stefan Institute, Department of Automatics, Biocybernetics, and Robotics, Jamova 39, 1000 Ljubljana, Slovenia;University Göttingen, Institute for Physics 3 - Biophysics, Bernstein Center for Computational Neuroscience, Friedrich-Hund-Platz 1, 37077 Göttingen, Germany
Venue:
Robotics and Autonomous Systems
Year:
2011

Citing 10
Cited 3

Introduction to Reinforcement Learning

Introduction to Reinforcement Learning
Robot Learning From Demonstration

ICML '97 Proceedings of the Fourteenth International Conference on Machine Learning
Learning CPG-based Biped Locomotion with a Policy Gradient Method: Application to a Humanoid Robot

International Journal of Robotics Research
2008 Special Issue: Reinforcement learning of motor skills with policy gradients

Neural Networks
Learning to reach by reinforcement learning using a receptive field based function approximation approach with continuous actions

Biological Cybernetics
Infinite-horizon policy-gradient estimation

Journal of Artificial Intelligence Research
Learning and generalization of motor skills by learning from demonstration

ICRA'09 Proceedings of the 2009 IEEE international conference on Robotics and Automation
Biologically-inspired dynamical systems for movement generation: automatic real-time goal adaptation and obstacle avoidance

ICRA'09 Proceedings of the 2009 IEEE international conference on Robotics and Automation
Dynamical System Modulation for Robot Learning via Kinesthetic Demonstrations

IEEE Transactions on Robotics
On Learning, Representing, and Generalizing a Task in a Humanoid Robot

IEEE Transactions on Systems, Man, and Cybernetics, Part B: Cybernetics

A robot learning from demonstration framework to perform force-based manipulation tasks

Intelligent Service Robotics
Reinforcement learning in robotics: A survey

International Journal of Robotics Research
Interaction learning for dynamic movement primitives used in cooperative robotic tasks

Robotics and Autonomous Systems

Quantified Score

Hi-index	0.00

Visualization

Abstract

When describing robot motion with dynamic movement primitives (DMPs), goal (trajectory endpoint), shape and temporal scaling parameters are used. In reinforcement learning with DMPs, usually goals and temporal scaling parameters are predefined and only the weights for shaping a DMP are learned. Many tasks, however, exist where the best goal position is not a priori known, requiring to learn it. Thus, here we specifically address the question of how to simultaneously combine goal and shape parameter learning. This is a difficult problem because learning of both parameters could easily interfere in a destructive way. We apply value function approximation techniques for goal learning and direct policy search methods for shape learning. Specifically, we use ''policy improvement with path integrals'' and ''natural actor critic'' for the policy search. We solve a learning-to-pour-liquid task in simulations as well as using a Pa10 robot arm. Results for learning from scratch, learning initialized by human demonstration, as well as for modifying the tool for the learned DMPs are presented. We observe that the combination of goal and shape learning is stable and robust within large parameter regimes. Learning converges quickly even in the presence of disturbances, which makes this combined method suitable for robotic applications.