When do differences matter? On-line feature extraction through cognitive economy

Authors:
David J. Finton
Affiliations:
Laboratory for Molecular and Computational Genomics, University of Wisconsin-Madison, Madison, WI 53706, USA
Venue:
Cognitive Systems Research
Year:
2005

Citing 10
Cited 1

The ART of Adaptive Pattern Recognition by a Self-Organizing Neural Network

Computer
Integrated architecture for learning, planning, and reacting based on approximating dynamic programming

Proceedings of the seventh international conference (1990) on Machine learning
A challenging set of control problems

Neural networks for control
Prioritized Sweeping: Reinforcement Learning with Less Data and Less Time

Machine Learning
A Sparse Sampling Algorithm for Near-Optimal Planning in Large Markov Decision Processes

Machine Learning
Variable Resolution Discretization in Optimal Control

Machine Learning
Adaptive Resolution Model-Free Reinforcement Learning: Decision Boundary Partitioning

ICML '00 Proceedings of the Seventeenth International Conference on Machine Learning
Reinforcement learning for dynamic robotic systems

Reinforcement learning for dynamic robotic systems
Cognitive economy and the role of representation in on-line learning

Cognitive economy and the role of representation in on-line learning
Reinforcement learning: a survey

Journal of Artificial Intelligence Research

Agent-based consumer learning in e-commerce

International Journal of Networking and Virtual Organisations

Quantified Score

Hi-index	0.00

Visualization

Abstract

For an intelligent agent to be truly autonomous, it must be able to adapt its representation to the requirements of its task as it interacts with the world. Most current approaches to on-line feature extraction are ad hoc; in contrast, this paper derives principled criteria for representational adequacy by applying the psychological principle of cognitive economy to reinforcement learning. The criteria are principled because they are based on an analysis of the amount of reward the agent forfeits when it generalizes over states. This analysis shows that action-value errors are sometimes irrelevant, and that the agent may optimize its performance with limited cognitive resources by grouping together states whose differences do not matter in its task. The paper presents an algorithm based on this analysis, incorporating an active form of Q-learning and partitioning continuous state-spaces by merging and splitting Voronoi regions. The experiments illustrate a new methodology for testing and comparing representations by means of learning curves. Results from the puck-on-a-hill task demonstrate the algorithm's ability to learn effective representations, superior to those produced by some other, well-known, methods.