Using background knowledge to speed reinforcement learning in physical agents

Authors:
Daniel Shapiro;Pat Langley;Ross Shachter
Affiliations:
Department of Management Science and Engineering, Stanford University, Stanford, CA;Institute for the Study of Learning and Expertise, 2164 Staunton Court, Palo Alto, CA;Department of Management Science and Engineering, Stanford University, Stanford, CA
Venue:
Proceedings of the fifth international conference on Autonomous agents
Year:
2001

Citing 15
Cited 6

Unified theories of cognition

Unified theories of cognition
A preliminary analysis of the Soar architecture as a basis for general intelligence

Artificial Intelligence
Controlling physical agents through reactive logic programming

Proceedings of the third annual conference on Autonomous Agents
Reinforcement learning with hierarchies of machines

NIPS '97 Proceedings of the 1997 conference on Advances in neural information processing systems 10
Convergence Results for Single-Step On-PolicyReinforcement-Learning Algorithms

Machine Learning
Learning Search Control Knowledge: An Explanation-Based Approach

Learning Search Control Knowledge: An Explanation-Based Approach
Chunking in Soar: The Anatomy of a General Learning Mechanism

Machine Learning
The MAXQ Method for Hierarchical Reinforcement Learning

ICML '98 Proceedings of the Fifteenth International Conference on Machine Learning
Intra-Option Learning about Temporally Abstract Actions

ICML '98 Proceedings of the Fifteenth International Conference on Machine Learning
Feudal Reinforcement Learning

Advances in Neural Information Processing Systems 5, [NIPS Conference]
The Dynamic Structure of Everyday Life

The Dynamic Structure of Everyday Life
Adaptive execution in complex dynamic worlds

Adaptive execution in complex dynamic worlds
Value-driven agents

Value-driven agents
Hierarchical reinforcement learning with the MAXQ value function decomposition

Journal of Artificial Intelligence Research
Reinforcement learning: a survey

Journal of Artificial Intelligence Research

Learning Hierarchical Skills from Observation

DS '02 Proceedings of the 5th International Conference on Discovery Science
Integrating Guidance into Relational Reinforcement Learning

Machine Learning
An Architecture for Persistent Reactive Behavior

AAMAS '04 Proceedings of the Third International Joint Conference on Autonomous Agents and Multiagent Systems - Volume 2
Knowledge of opposite actions for reinforcement learning

Applied Soft Computing
SmartGantt - An intelligent system for real time rescheduling based on relational reinforcement learning

Expert Systems with Applications: An International Journal
Acquisition of hierarchical reactive skills in a unified cognitive architecture

Cognitive Systems Research

Quantified Score

Hi-index	0.00

Visualization

Abstract

This paper describes Icarus, an agent architecture that embeds a hierarchical reinforcement learning algorithm within a language for specifying agent behavior. An Icarus program expresses an approximately correct theory about how to behave with options at varying levels of detail, while the Icarus agent determines the best options by learning from experience. We describe Icarus and its learning algorithm, then report on two experiments in a vehicle control domain. The first examines the benefit of new distinctions about state, whereas the second explores the impact of added plan structure. We show that background knowledge increases learning rate and asymptotic performance, and decreases plan size by three orders of magnitude, relative to the typical formulation of the learning problem in our test domain.