Transfer learning in real-time strategy games using hybrid CBR/RL

Authors:
Manu Sharma;Michael Holmes;Juan Santamaria;Arya Irani;Charles Isbell;Ashwin Ram
Affiliations:
College of Computing, Georgia Institute of Technology;College of Computing, Georgia Institute of Technology;College of Computing, Georgia Institute of Technology;College of Computing, Georgia Institute of Technology;College of Computing, Georgia Institute of Technology;College of Computing, Georgia Institute of Technology
Venue:
IJCAI'07 Proceedings of the 20th international joint conference on Artifical intelligence
Year:
2007

Citing 8
Cited 24

Case-based reasoning

Case-based reasoning
Case-based reasoning: foundational issues, methodological variations, and system approaches

AI Communications
Experiments with reinforcement learning in problems with continuous state and action spaces

Adaptive Behavior
Case-Based Reasoning: Experiences, Lessons and Future Directions

Case-Based Reasoning: Experiences, Lessons and Future Directions
Introduction to Reinforcement Learning

Introduction to Reinforcement Learning
Learning to Predict by the Methods of Temporal Differences

Machine Learning
Learning to win: case-based plan selection in a real-time strategy game

ICCBR'05 Proceedings of the 6th international conference on Case-Based Reasoning Research and Development
CBR for state value function approximation in reinforcement learning

ICCBR'05 Proceedings of the 6th international conference on Case-Based Reasoning Research and Development

Autonomous transfer for reinforcement learning

Proceedings of the 7th international joint conference on Autonomous agents and multiagent systems - Volume 1
Case-Based Planning and Execution for Real-Time Strategy Games

ICCBR '07 Proceedings of the 7th international conference on Case-Based Reasoning: Case-Based Reasoning Research and Development
Recognizing the Enemy: Combining Reinforcement Learning with Strategy Selection Using Case-Based Reasoning

ECCBR '08 Proceedings of the 9th European conference on Advances in Case-Based Reasoning
Situation Assessment for Plan Retrieval in Real-Time Strategy Games

ECCBR '08 Proceedings of the 9th European conference on Advances in Case-Based Reasoning
Forgetting Reinforced Cases

ECCBR '08 Proceedings of the 9th European conference on Advances in Case-Based Reasoning
Case-Based Reasoning in Transfer Learning

ICCBR '09 Proceedings of the 8th International Conference on Case-Based Reasoning: Case-Based Reasoning Research and Development
Improving Reinforcement Learning by Using Case Based Heuristics

ICCBR '09 Proceedings of the 8th International Conference on Case-Based Reasoning: Case-Based Reasoning Research and Development
Spatial Event Prediction by Combining Value Function Approximation and Case-Based Reasoning

ICCBR '09 Proceedings of the 8th International Conference on Case-Based Reasoning: Case-Based Reasoning Research and Development
Measuring the level of transfer learning by an AP physics problem-solver

AAAI'07 Proceedings of the 22nd national conference on Artificial intelligence - Volume 1
Analogical model formulation for transfer learning in AP Physics

Artificial Intelligence
Discovering Abstract Concepts to Aid Cross-Map Transfer for a Learning Agent

DS '09 Proceedings of the 12th International Conference on Discovery Science
Transfer Learning for Reinforcement Learning Domains: A Survey

The Journal of Machine Learning Research
Case-Based Multiagent Reinforcement Learning: Cases as Heuristics for Selection of Actions

Proceedings of the 2010 conference on ECAI 2010: 19th European Conference on Artificial Intelligence
Time-based reward shaping in real-time strategy games

ADMI'10 Proceedings of the 6th international conference on Agents and data mining interaction
Adaptive two-player videogames

Expert Systems with Applications: An International Journal
Determining solution space characteristics for real-time strategy games and characterizing winning strategies

International Journal of Computer Games Technology
Reducing the memory footprint of temporal difference learning over finitely many states by using case-based generalization

ICCBR'10 Proceedings of the 18th international conference on Case-Based Reasoning Research and Development
Imitating inscrutable enemies: learning from stochastic policy observation, retrieval and reuse

ICCBR'10 Proceedings of the 18th international conference on Case-Based Reasoning Research and Development
A data-driven approach for resource gathering in real-time strategy games

ADMI'11 Proceedings of the 7th international conference on Agents and Data Mining Interaction
Using cases as heuristics in reinforcement learning: a transfer learning application

IJCAI'11 Proceedings of the Twenty-Second international joint conference on Artificial Intelligence - Volume Volume Two
Personalised gaming: a motivation and overview of literature

Proceedings of The 8th Australasian Conference on Interactive Entertainment: Playing the System
Case-based strategies in computer poker

AI Communications
Cognitive agents for microscopic traffic simulations in virtual environments

ICEC'12 Proceedings of the 11th international conference on Entertainment Computing
Safe exploration of state and action spaces in reinforcement learning

Journal of Artificial Intelligence Research

Quantified Score

Hi-index	0.00

Visualization

Abstract

The goal of transfer learning is to use the knowledge acquired in a set of source tasks to improve performance in a related but previously unseen target task. In this paper, we present a multilayered architecture named CAse-Based Reinforcement Learner (CARL). It uses a novel combination of Case-Based Reasoning (CBR) and Reinforcement Learning (RL) to achieve transfer while playing against the Game AI across a variety of scenarios in MadRTSTM, a commercial Real Time Strategy game. Our experiments demonstrate that CARL not only performs well on individual tasks but also exhibits significant performance gains when allowed to transfer knowledge from previous tasks.