Technical Note: \cal Q-Learning
Machine Learning
A teaching method for reinforcement learning
ML92 Proceedings of the ninth international workshop on Machine learning
The role of emotion in believable agents
Communications of the ACM
Collaborative interface agents
AAAI '94 Proceedings of the twelfth national conference on Artificial intelligence (vol. 1)
Virtual petz (video session): a hybrid approach to creating autonomous, lifelike dogz and catz
AGENTS '98 Proceedings of the second international conference on Autonomous agents
A social reinforcement learning agent
Proceedings of the fifth international conference on Autonomous agents
Designing Sociable Robots
Integrated learning for interactive synthetic characters
Proceedings of the 29th annual conference on Computer graphics and interactive techniques
Reinforcement Learning in the Multi-Robot Domain
Autonomous Robots
Less is More: Active Learning with Support Vector Machines
ICML '00 Proceedings of the Seventeenth International Conference on Machine Learning
Natural methods for robot task learning: instructive demonstrations, generalization and practice
AAMAS '03 Proceedings of the second international joint conference on Autonomous agents and multiagent systems
Lifelong Robot Learning
Old tricks, new dogs: ethology and interactive creatures
Old tricks, new dogs: ethology and interactive creatures
Extracting knowledge about users' activities from raw workstation contents
AAAI'06 Proceedings of the 21st national conference on Artificial intelligence - Volume 1
AAAI'05 Proceedings of the 20th national conference on Artificial intelligence - Volume 2
The lumière project: Bayesian user modeling for inferring the goals and needs of software users
UAI'98 Proceedings of the Fourteenth conference on Uncertainty in artificial intelligence
Connection Science - Social Learning in Embodied Agents
Teaching robot companions: the role of scaffolding and event structuring
Connection Science - Social Learning in Embodied Agents
Learning about objects with human teachers
Proceedings of the 4th ACM/IEEE international conference on Human robot interaction
A Platform System for Developing a Collaborative Mutually Adaptive Agent
IEA/AIE '09 Proceedings of the 22nd International Conference on Industrial, Engineering and Other Applications of Applied Intelligent Systems: Next-Generation Applied Intelligence
Actively Adaptive Agent for Human-Agent Collaborative Task
AMT '09 Proceedings of the 5th International Conference on Active Media Technology
Transparent active learning for robots
Proceedings of the 5th ACM/IEEE international conference on Human-robot interaction
Learning Visual Object Categories for Robot Affordance Prediction
International Journal of Robotics Research
A Human-Robot Collaborative Reinforcement Learning Algorithm
Journal of Intelligent and Robotic Systems
Exploiting social partners in robot learning
Autonomous Robots
TellMe: learning procedures from tutorial instruction
Proceedings of the 16th international conference on Intelligent user interfaces
A formal framework for combining natural instruction and demonstration for end-user programming
Proceedings of the 16th international conference on Intelligent user interfaces
Robots that express emotion elicit better human teaching
Proceedings of the 6th international conference on Human-robot interaction
Robot self-initiative and personalization by learning through repeated interactions
Proceedings of the 6th international conference on Human-robot interaction
Active adaptation in human-agent collaborative interaction
Journal of Intelligent Information Systems
Towards understanding how humans teach robots
UMAP'11 Proceedings of the 19th international conference on User modeling, adaption, and personalization
Formation conditions of mutual adaptation in human-agent collaborative interaction
Applied Intelligence
Style by demonstration: teaching interactive movement style to robots
Proceedings of the 2012 ACM international conference on Intelligent User Interfaces
Human behavior understanding for robotics
HBU'12 Proceedings of the Third international conference on Human Behavior Understanding
Compliant skills acquisition and multi-optima policy search with EM-based reinforcement learning
Robotics and Autonomous Systems
Learning non-myopically from human-generated reward
Proceedings of the 2013 international conference on Intelligent user interfaces
Teaching agents with human feedback: a demonstration of the TAMER framework
Proceedings of the companion publication of the 2013 international conference on Intelligent user interfaces companion
Machine learning for interactive systems and robots: a brief introduction
Proceedings of the 2nd Workshop on Machine Learning for Interactive Systems: Bridging the Gap Between Perception, Action and Communication
Social contracts and human-computer interaction with simulated adapting agents
Adaptive Behavior - Animals, Animats, Software Agents, Robots, Adaptive Systems
Hi-index | 0.00 |
While Reinforcement Learning (RL) is not traditionally designed for interactive supervisory input from a human teacher, several works in both robot and software agents have adapted it for human input by letting a human trainer control the reward signal. In this work, we experimentally examine the assumption underlying these works, namely that the human-given reward is compatible with the traditional RL reward signal. We describe an experimental platform with a simulated RL robot and present an analysis of real-time human teaching behavior found in a study in which untrained subjects taught the robot to perform a new task. We report three main observations on how people administer feedback when teaching a Reinforcement Learning agent: (a) they use the reward channel not only for feedback, but also for future-directed guidance; (b) they have a positive bias to their feedback, possibly using the signal as a motivational channel; and (c) they change their behavior as they develop a mental model of the robotic learner. Given this, we made specific modifications to the simulated RL robot, and analyzed and evaluated its learning behavior in four follow-up experiments with human trainers. We report significant improvements on several learning measures. This work demonstrates the importance of understanding the human-teacher/robot-learner partnership in order to design algorithms that support how people want to teach and simultaneously improve the robot's learning behavior.