A survey of robot learning from demonstration

Authors:
Brenna D. Argall;Sonia Chernova;Manuela Veloso;Brett Browning
Affiliations:
Robotics Institute, Carnegie Mellon University, Pittsburgh, PA 15213, USA;Computer Science Department, Carnegie Mellon University, Pittsburgh, PA 15213, USA;Computer Science Department, Carnegie Mellon University, Pittsburgh, PA 15213, USA;Robotics Institute, Carnegie Mellon University, Pittsburgh, PA 15213, USA
Venue:
Robotics and Autonomous Systems
Year:
2009

Citing 37
Cited 85

Learning to fly

ML92 Proceedings of the ninth international workshop on Machine learning
Locally Weighted Learning

Artificial Intelligence Review - Special issue on lazy learning
Learning agents for uncertain environments (extended abstract)

COLT' 98 Proceedings of the eleventh annual conference on Computational learning theory
Learning procedural knowledge through observation

Proceedings of the 1st international conference on Knowledge capture
Reinforcement Learning

Reinforcement Learning
Robot Learning From Demonstration

ICML '97 Proceedings of the Fourteenth International Conference on Machine Learning
Locally Weighted Projection Regression: Incremental Real Time Learning in High Dimensional Space

ICML '00 Proceedings of the Seventeenth International Conference on Machine Learning
Imitation in animals and artifacts

Imitation in animals and artifacts
Natural methods for robot task learning: instructive demonstrations, generalization and practice

AAMAS '03 Proceedings of the second international joint conference on Autonomous agents and multiagent systems
Learning Movement Sequences from Demonstration

ICDL '02 Proceedings of the 2nd International Conference on Development and Learning
On integrating apprentice learning and reinforcement learning

On integrating apprentice learning and reinforcement learning
Making reinforcement learning work on real robots

Making reinforcement learning work on real robots
An introduction to variable and feature selection

The Journal of Machine Learning Research
Apprenticeship learning via inverse reinforcement learning

ICML '04 Proceedings of the twenty-first international conference on Machine learning
Learning from observation using primitives

Learning from observation using primitives
Exploration and apprenticeship learning in reinforcement learning

ICML '05 Proceedings of the 22nd international conference on Machine learning
Common metrics for human-robot interaction

Proceedings of the 1st ACM SIGCHI/SIGART conference on Human-robot interaction
Teaching robots by moulding behavior and scaffolding the environment

Proceedings of the 1st ACM SIGCHI/SIGART conference on Human-robot interaction
Evaluation of robot imitation attempts: comparison of the system's and the human's perspectives

Proceedings of the 1st ACM SIGCHI/SIGART conference on Human-robot interaction
Constructive Incremental Learning from Only Local Information

Neural Computation
Maximum margin planning

ICML '06 Proceedings of the 23rd international conference on Machine learning
Interactive robot task training through dialog and demonstration

Proceedings of the ACM/IEEE international conference on Human-robot interaction
Learning by demonstration with critique from a human teacher

Proceedings of the ACM/IEEE international conference on Human-robot interaction
Incremental learning of gestures by imitation in a humanoid robot

Proceedings of the ACM/IEEE international conference on Human-robot interaction
A multi-agent system for programming robots by human demonstration

Integrated Computer-Aided Engineering
Confidence-based policy learning from demonstration using Gaussian mixture models

Proceedings of the 6th international joint conference on Autonomous agents and multiagent systems
Multi-thresholded approach to demonstration selection for interactive robot learning

Proceedings of the 3rd ACM/IEEE international conference on Human robot interaction
Efficient training of artificial neural networks for autonomous navigation

Neural Computation
Natural Actor-Critic

Neurocomputing
Teaching multi-robot coordination using demonstration of communication and state sharing

Proceedings of the 7th international joint conference on Autonomous agents and multiagent systems - Volume 3
Teaching sequential tasks with repetition through demonstration

Proceedings of the 7th international joint conference on Autonomous agents and multiagent systems - Volume 3
Skill acquisition and use for a dynamically-balancing soccer robot

AAAI'04 Proceedings of the 19th national conference on Artifical intelligence
Reinforcement learning with human teachers: evidence of feedback and guidance with implications for learning performance

AAAI'06 Proceedings of the 21st national conference on Artificial intelligence - Volume 1
Maximum entropy inverse reinforcement learning

AAAI'08 Proceedings of the 23rd national conference on Artificial intelligence - Volume 3
Toward automatic robot programming: learning human skill fromvisual data

IEEE Transactions on Systems, Man, and Cybernetics, Part B: Cybernetics
Visual learning by imitation with motor representations

IEEE Transactions on Systems, Man, and Cybernetics, Part B: Cybernetics
Correspondence Mapping Induced State and Action Metrics for Robotic Imitation

IEEE Transactions on Systems, Man, and Cybernetics, Part B: Cybernetics

Learning to search: Functional gradient techniques for imitation learning

Autonomous Robots
Constraint specification by example in a Meta-CASE tool

Proceedings of the doctoral symposium for ESEC/FSE on Doctoral symposium
Interactive policy learning through confidence-based autonomy

Journal of Artificial Intelligence Research
Unsupervised simultaneous learning of gestures, actions and their associations for human-robot interaction

IROS'09 Proceedings of the 2009 IEEE/RSJ international conference on Intelligent robots and systems
Effective robot task learning by focusing on task-relevant objects

IROS'09 Proceedings of the 2009 IEEE/RSJ international conference on Intelligent robots and systems
Programming-by-Demonstration of reaching motions-A next-state-planner approach

Robotics and Autonomous Systems
Combining manual feedback with subsequent MDP reward signals for reinforcement learning

Proceedings of the 9th International Conference on Autonomous Agents and Multiagent Systems: volume 1 - Volume 1
Using training regimens to teach expanding function approximators

Proceedings of the 9th International Conference on Autonomous Agents and Multiagent Systems: volume 1 - Volume 1
An effective personal mobile robot agent through symbiotic human-robot interaction

Proceedings of the 9th International Conference on Autonomous Agents and Multiagent Systems: volume 1 - Volume 1
Learning multirobot joint action plans from simultaneous task execution demonstrations

Proceedings of the 9th International Conference on Autonomous Agents and Multiagent Systems: volume 1 - Volume 1
Cultivating desired behaviour: policy teaching via environment-dynamics tweaks

Proceedings of the 9th International Conference on Autonomous Agents and Multiagent Systems: volume 1 - Volume 1
An empirical study of specification by example in a software engineering tool

Proceedings of the 2010 ACM-IEEE International Symposium on Empirical Software Engineering and Measurement
A Human-Robot Collaborative Reinforcement Learning Algorithm

Journal of Intelligent and Robotic Systems
Learning inverse kinematics for pose-constraint bi-manual movements

SAB'10 Proceedings of the 11th international conference on Simulation of adaptive behavior: from animals to animats
Learning from demonstration using MDP induced metrics

ECML PKDD'10 Proceedings of the 2010 European conference on Machine learning and knowledge discovery in databases: Part II
The well-designed logical robot: Learning and experience from observations to the Situation Calculus

Artificial Intelligence
Robot programming by demonstration

SIMPAR'10 Proceedings of the Second international conference on Simulation, modeling, and programming for autonomous robots
On internal knowledge representation for programming mobile robots by demonstration

ICSR'10 Proceedings of the Second international conference on Social robotics
Teacher feedback to scaffold and refine demonstrated motion primitives on a mobile robot

Robotics and Autonomous Systems
Policy adaptation with tactile feedback

Proceedings of the 6th international conference on Human-robot interaction
Learning from failure: extended abstract

Proceedings of the 6th international conference on Human-robot interaction
Human and robot perception in large-scale learning from demonstration

Proceedings of the 6th international conference on Human-robot interaction
Robot self-initiative and personalization by learning through repeated interactions

Proceedings of the 6th international conference on Human-robot interaction
Learning search heuristics for finding objects in structured environments

Robotics and Autonomous Systems
Learning from demonstration in robots: Experimental comparison of neural architectures

Robotics and Computer-Integrated Manufacturing
Teaching a humanoid robot to draw `Shapes'

Autonomous Robots
Towards understanding how humans teach robots

UMAP'11 Proceedings of the 19th international conference on User modeling, adaption, and personalization
Integrating reinforcement learning with human demonstrations of varying ability

The 10th International Conference on Autonomous Agents and Multiagent Systems - Volume 2
Learning by demonstration in repeated stochastic games

The 10th International Conference on Autonomous Agents and Multiagent Systems - Volume 3
Towards robot incremental learning constraints from comparative demonstration

The 10th International Conference on Autonomous Agents and Multiagent Systems - Volume 3
On-line regression algorithms for learning mechanical models of robots: A survey

Robotics and Autonomous Systems
Interactive robot trajectory planning and simulation using Augmented Reality

Robotics and Computer-Integrated Manufacturing
An overview of 3D object grasp synthesis algorithms

Robotics and Autonomous Systems
Designing robot learners that ask good questions

HRI '12 Proceedings of the seventh annual ACM/IEEE international conference on Human-Robot Interaction
Policy transformation for learning from demonstration

HRI '12 Proceedings of the seventh annual ACM/IEEE international conference on Human-Robot Interaction
A practical comparison of three robot learning from demonstration algorithms

HRI '12 Proceedings of the seventh annual ACM/IEEE international conference on Human-Robot Interaction
Trajectories and keyframes for kinesthetic teaching: a human-robot interaction perspective

HRI '12 Proceedings of the seventh annual ACM/IEEE international conference on Human-Robot Interaction
Is Someone in this Office Available to Help Me?

Journal of Intelligent and Robotic Systems
Robot learning from demonstration by constructing skill trees

International Journal of Robotics Research
Novel Applications of VR: Virtual Network Marathon with immersion, scientificalness, competitiveness, adaptability and learning

Computers and Graphics
Situated learning of visual robot behaviors

ICIRA'11 Proceedings of the 4th international conference on Intelligent Robotics and Applications - Volume Part I
A nonparametric Bayesian approach toward robot learning by demonstration

Robotics and Autonomous Systems
Tactile Guidance for Policy Adaptation

Foundations and Trends in Robotics
Improving biped walk stability with complementary corrective demonstration

Autonomous Robots
Reinforcement learning with limited reinforcement: Using Bayes risk for active learning in POMDPs

Artificial Intelligence
Automatic state abstraction from demonstration

IJCAI'11 Proceedings of the Twenty-Second international joint conference on Artificial Intelligence - Volume Volume Two
Imitation learning in relational domains: a functional-gradient boosting approach

IJCAI'11 Proceedings of the Twenty-Second international joint conference on Artificial Intelligence - Volume Volume Two
Automatic task decomposition and state abstraction from demonstration

Proceedings of the 11th International Conference on Autonomous Agents and Multiagent Systems - Volume 1
Multi Robot Learning by Demonstration

Proceedings of the 11th International Conference on Autonomous Agents and Multiagent Systems - Volume 3
Strategy-Based learning through communication with humans

KES-AMSTA'12 Proceedings of the 6th KES international conference on Agent and Multi-Agent Systems: technologies and applications
Bayesian nonparametric inverse reinforcement learning

ECML PKDD'12 Proceedings of the 2012 European conference on Machine Learning and Knowledge Discovery in Databases - Volume Part II
Human behavior understanding for robotics

HBU'12 Proceedings of the Third international conference on Human Behavior Understanding
Semantic world modeling using probabilistic multiple hypothesis anchoring

Robotics and Autonomous Systems
A robot learning from demonstration framework to perform force-based manipulation tasks

Intelligent Service Robotics
Fast motion planning from experience: trajectory prediction for speeding up movement generation

Autonomous Robots
Safe exploration of state and action spaces in reinforcement learning

Journal of Artificial Intelligence Research
Compliant skills acquisition and multi-optima policy search with EM-based reinforcement learning

Robotics and Autonomous Systems
Human-robot cross-training: computational formulation, modeling and evaluation of a human team training strategy

Proceedings of the 8th ACM/IEEE international conference on Human-robot interaction
Legible user input for intent prediction

Proceedings of the 8th ACM/IEEE international conference on Human-robot interaction
Execution memory for grounding and coordination

Proceedings of the 8th ACM/IEEE international conference on Human-robot interaction
Anthropomorphism in the factory: a paradigm change?

Proceedings of the 8th ACM/IEEE international conference on Human-robot interaction
Legibility and predictability of robot motion

Proceedings of the 8th ACM/IEEE international conference on Human-robot interaction
Learning non-myopically from human-generated reward

Proceedings of the 2013 international conference on Intelligent user interfaces
Using informative behavior to increase engagement in the tamer framework

Proceedings of the 2013 international conference on Autonomous agents and multi-agent systems
Teaching on a budget: agents advising agents in reinforcement learning

Proceedings of the 2013 international conference on Autonomous agents and multi-agent systems
Machine learning for interactive systems and robots: a brief introduction

Proceedings of the 2nd Workshop on Machine Learning for Interactive Systems: Bridging the Gap Between Perception, Action and Communication
Learning to select and generalize striking movements in robot table tennis

International Journal of Robotics Research
An approach to universal interaction on the case of knowledge transfer

UAHCI'13 Proceedings of the 7th international conference on Universal Access in Human-Computer Interaction: user and context diversity - Volume 2
Probabilistic model-based imitation learning

Adaptive Behavior - Animals, Animats, Software Agents, Robots, Adaptive Systems
Reinforcement learning in robotics: A survey

International Journal of Robotics Research
Bayesian nonparametric feature construction for inverse reinforcement learning

IJCAI'13 Proceedings of the Twenty-Third international joint conference on Artificial Intelligence
On teaching collaboration to a team of autonomous agents via imitation

IJCAI'13 Proceedings of the Twenty-Third international joint conference on Artificial Intelligence
Real-time human-robot interactive coaching system with full-body control interface

Robot Soccer World Cup XV
Learning partial ordering constraints from a single demonstration

Proceedings of the 2014 ACM/IEEE international conference on Human-robot interaction
Reasons for singularity in robot teleoperation

Proceedings of the 2014 ACM/IEEE international conference on Human-robot interaction
Fast damage recovery in robotics with the T-resilience algorithm

International Journal of Robotics Research
Learning by observation of agent software images

Journal of Artificial Intelligence Research
Hardware opposition-based PSO applied to mobile robot controllers

Engineering Applications of Artificial Intelligence
The actor's view of automated planning and acting: A position paper

Artificial Intelligence
Autonomously learning to visually detect where manipulation will succeed

Autonomous Robots
Teaching robots to cooperate with humans in dynamic manipulation tasks based on multi-modal human-in-the-loop approach

Autonomous Robots
Socially guided intrinsic motivation for robot learning of motor skills

Autonomous Robots
Object-object interaction affordance learning

Robotics and Autonomous Systems
Robotics and artificial intelligence: A perspective on deliberation functions

AI Communications - ECAI 2012 Turing and Anniversary Track
A comparison between a communication-based and a data mining-based learning approach for agents

Intelligent Decision Technologies

Quantified Score

Hi-index	0.00

Visualization

Abstract

We present a comprehensive survey of robot Learning from Demonstration (LfD), a technique that develops policies from example state to action mappings. We introduce the LfD design choices in terms of demonstrator, problem space, policy derivation and performance, and contribute the foundations for a structure in which to categorize LfD research. Specifically, we analyze and categorize the multiple ways in which examples are gathered, ranging from teleoperation to imitation, as well as the various techniques for policy derivation, including matching functions, dynamics models and plans. To conclude we discuss LfD limitations and related promising areas for future research.