Incremental learning by message passing in hierarchical temporal memory

Authors:
Davide Maltoni;Erik M. Rehn
Affiliations:
Biometric System Laboratory, DEIS, University of Bologna, Italy;Bernstein Center for Computational Neuroscience, Berlin, Germany
Venue:
ANNPR'12 Proceedings of the 5th INNS IAPR TC 3 GIRPR conference on Artificial Neural Networks in Pattern Recognition
Year:
2012

Citing 3
Cited 0

Probabilistic reasoning in intelligent systems: networks of plausible inference

Probabilistic reasoning in intelligent systems: networks of plausible inference
How the brain might work: a hierarchical and temporal model for learning and recognition

How the brain might work: a hierarchical and temporal model for learning and recognition
Learning Deep Architectures for AI

Foundations and Trends® in Machine Learning

Quantified Score

Hi-index	0.00

Visualization

Abstract

Hierarchical Temporal Memory is a biologically-inspired framework that can be used to learn invariant representations of patterns. Classical HTM learning is mainly unsupervised and once training is completed the network structure is frozen, thus making further training quite critical. In this paper we develop a novel technique for HTM (incremental) supervised learning based on error minimization. We prove that error backpropagation can be naturally and elegantly implemented through native HTM message passing based on Belief Propagation. Our experimental results show that a two stage training composed by unsupervised pre-training + supervised refinement is very effective. This is in line with recent findings on other deep architectures.