XCS with adaptive action mapping

Authors:
Masaya Nakata;Pier Luca Lanzi;Keiki Takadama
Affiliations:
Department of Informatics, The University of Electro-Communications, Tokyo, Japan;Dipartimento di Elettronica e Informazione, Politecnico di Milano, Milano, Italy;Department of Informatics, The University of Electro-Communications, Tokyo, Japan
Venue:
SEAL'12 Proceedings of the 9th international conference on Simulated Evolution and Learning
Year:
2012

Citing 8
Cited 1

Genetic Algorithms in Search, Optimization and Machine Learning

Genetic Algorithms in Search, Optimization and Machine Learning
Introduction to Reinforcement Learning

Introduction to Reinforcement Learning
Accuracy-based learning classifier systems: models, analysis and applications to classification tasks

Evolutionary Computation
Classifier fitness based on accuracy

Evolutionary Computation
An analysis of generalization in the xcs classifier system

Evolutionary Computation
Tournament selection: stable fitness pressure in XCS

GECCO'03 Proceedings of the 2003 international conference on Genetic and evolutionary computation: PartII
Toward a theory of generalization and learning in XCS

IEEE Transactions on Evolutionary Computation
Gradient descent methods in learning classifier systems: improving XCS performance in multistep problems

IEEE Transactions on Evolutionary Computation

Selection strategy for XCS with adaptive action mapping

Proceedings of the 15th annual conference on Genetic and evolutionary computation

Quantified Score

Hi-index	0.00

Visualization

Abstract

The XCS classifier system evolves solutions that represent complete mappings from state-action pairs to expected returns therefore, in every possible situation, XCS can predict the value of all the available actions. Such complete mapping is sometimes considered redundant as most of the applications (like for instance, classification), usually focus only on the best action. In this paper, we introduce an extension of XCS with an adaptive (state-action) mapping mechanism (or XCSAM) that evolves solutions focused actions with the largest returns. While UCS evolves solutions focused on the best available action but can only solve supervised classification problems, our system can solve both supervised and multi-step problems and, in addition, it can adapt the size of the mapping to the problems: Initially, XCSAM starts building a complete mapping and then it slowly tries to focus on the best actions available. If the problem admits only one optimal action in each niche, XCSAM tends to focus on such an action as the evolution proceeds. If more actions with the same return are available, XCSAM tends to evolve a mapping that includes all of them. We applied XCSAM both to supervised problems (the Boolean multiplexer) and to multi-step maze-like problems. Our experimental results show that XCSAM can reach optimal performance but requires smaller populations than XCS as it evolves solutions focused on the best actions available for each subproblem.