Confidence-based robot policy learning from demonstration

• Authors: Manuela Veloso;Sonia Chernova
• Venue: Confidence-based robot policy learning from demonstration
• Year: 2009

The problem of learning a policy, a task representation mapping from world states to actions, lies at the heart of many robotic applications. One approach to acquiring a task policy is learning from demonstration, an interactive technique in which a robot learns a policy based on example state to action mappings provided by a human teacher. This thesis introduces Confidence-Based Autonomy, a mixed-initiative single robot demonstration learning algorithm that enables the robot and teacher to jointly control the learning process and selection of demonstration training data. The robot to identifies the need for and requests demonstrations for specific parts of the state space based on confidence thresholds characterizing the uncertainty of the learned policy. The robot's demonstration requests are complemented by the teacher's ability to provide supplementary corrective demonstrations in error cases. An additional algorithmic component enables choices between multiple equally applicable actions to be represented explicitly within the robot's policy through the creation of option classes. Based on the single-robot Confidence-Based Autonomy algorithm, this thesis introduces a task and platform independent multi-robot demonstration learning framework for teaching multiple robots. Building upon this framework, we formalize three approaches to teaching emergent collaborative behavior based on different information sharing strategies. We provide detailed evaluations of all algorithms in multiple simulated and robotic domains, and present a case study analysis of the scalability of the presented techniques using up to seven robots.