Abstraction and generalization in reinforcement learning: a summary and framework

Authors:
Marc Ponsen;Matthew E. Taylor;Karl Tuyls
Affiliations:
Universiteit Maastricht, Maastricht, The Netherlands;The University of Southern California, Los Angeles, CA;Universiteit Maastricht, Maastricht, The Netherlands
Venue:
ALA'09 Proceedings of the Second international conference on Adaptive and Learning Agents
Year:
2009

Citing 34
Cited 1

Intelligence without representation

Artificial Intelligence
Dyna, an integrated architecture for learning, planning, and reacting

ACM SIGART Bulletin
A theory of abstraction

Artificial Intelligence
TD-Gammon, a self-teaching backgammon program, achieves master-level play

Neural Computation
Multitask Learning

Machine Learning - Special issue on inductive transfer
Between MDPs and semi-MDPs: a framework for temporal abstraction in reinforcement learning

Artificial Intelligence
Relational reinforcement learning

Machine Learning - Special issue on inducive logic programming
Markov Decision Processes: Discrete Stochastic Dynamic Programming

Markov Decision Processes: Discrete Stochastic Dynamic Programming
Brains, Behavior and Robotics

Brains, Behavior and Robotics
Introduction to Reinforcement Learning

Introduction to Reinforcement Learning
Near-Optimal Reinforcement Learning in Polynominal Time

ICML '98 Proceedings of the Fifteenth International Conference on Machine Learning
An Overview of MAXQ Hierarchical Reinforcement Learning

SARA '02 Proceedings of the 4th International Symposium on Abstraction, Reformulation, and Approximation
Model Minimization in Hierarchical Reinforcement Learning

Proceedings of the 5th International Symposium on Abstraction, Reformulation and Approximation
A Multiagent Variant of Dyna-Q

ICMAS '00 Proceedings of the Fourth International Conference on MultiAgent Systems (ICMAS-2000)
Recent Advances in Hierarchical Reinforcement Learning

Discrete Event Dynamic Systems
R-max - a general polynomial time algorithm for near-optimal reinforcement learning

The Journal of Machine Learning Research
Tree-Based Batch Mode Reinforcement Learning

The Journal of Machine Learning Research
Autonomous shaping: knowledge transfer in reinforcement learning

ICML '06 Proceedings of the 23rd international conference on Machine learning
Cross-domain transfer for reinforcement learning

Proceedings of the 24th international conference on Machine learning
Transfer Learning via Inter-Task Mappings for Temporal Difference Learning

The Journal of Machine Learning Research
Proto-value Functions: A Laplacian Framework for Learning Representation and Control in Markov Decision Processes

The Journal of Machine Learning Research
Transfer of samples in batch reinforcement learning

Proceedings of the 25th international conference on Machine learning
Transferring Instances for Model-Based Reinforcement Learning

ECML PKDD '08 Proceedings of the European conference on Machine Learning and Knowledge Discovery in Databases - Part II
Curriculum learning

ICML '09 Proceedings of the 26th Annual International Conference on Machine Learning
Transfer via soft homomorphisms

Proceedings of The 8th International Conference on Autonomous Agents and Multiagent Systems - Volume 2
Learning with whom to communicate using relational reinforcement learning

Proceedings of The 8th International Conference on Autonomous Agents and Multiagent Systems - Volume 2
Using Homomorphisms to transfer options across continuous reinforcement learning domains

AAAI'06 Proceedings of the 21st national conference on Artificial intelligence - Volume 1
Building portable options: skill transfer in reinforcement learning

IJCAI'07 Proceedings of the 20th international joint conference on Artifical intelligence
Transfer Learning for Reinforcement Learning Domains: A Survey

The Journal of Machine Learning Research
Model-based exploration in continuous state spaces

SARA'07 Proceedings of the 7th International conference on Abstraction, reformulation, and approximation
Learning relational options for inductive transfer in relational reinforcement learning

ILP'07 Proceedings of the 17th international conference on Inductive logic programming
Relational macros for transfer in reinforcement learning

ILP'07 Proceedings of the 17th international conference on Inductive logic programming
Model minimization in Markov decision processes

AAAI'97/IAAI'97 Proceedings of the fourteenth national conference on artificial intelligence and ninth conference on Innovative applications of artificial intelligence
Keepaway soccer: from machine learning testbed to benchmark

RoboCup 2005

Curriculum learning for motor skills

Canadian AI'12 Proceedings of the 25th Canadian conference on Advances in Artificial Intelligence

Quantified Score

Hi-index	0.00

Visualization

Abstract

In this paper we survey the basics of reinforcement learning, generalization and abstraction. We start with an introduction to the fundamentals of reinforcement learning and motivate the necessity for generalization and abstraction. Next we summarize the most important techniques available to achieve both generalization and abstraction in reinforcement learning. We discuss basic function approximation techniques and delve into hierarchical, relational and transfer learning. All concepts and techniques are illustrated with examples.