A Sparse and Locally Shift Invariant Feature Extractor Applied to Document Images

Authors:
M. Ranzato;Y. LeCun
Affiliations:
New York University - New York, NY;New York University - New York, NY
Venue:
ICDAR '07 Proceedings of the Ninth International Conference on Document Analysis and Recognition - Volume 02
Year:
2007

Citing 0
Cited 3

Mimicking Go Experts with Convolutional Neural Networks

ICANN '08 Proceedings of the 18th international conference on Artificial Neural Networks, Part II
Fast and robust learning by reinforcement signals: Explorations in the insect brain

Neural Computation
Learning Deep Architectures for AI

Foundations and Trends® in Machine Learning

Quantified Score

Hi-index	0.00

Visualization

Abstract

We describe an unsupervised learning algorithm for ex- tracting sparse and locally shift-invariant features. We also devise a principled procedure for learning hierarchies of in- variant features. Each feature detector is composed of a set of trainable convolutional filters followed by a max-pooling layer over non-overlapping windows, and a point-wise sig- moid non-linearity. A second stage of more invariant fea- tures is fed with patches provided by the first stage feature extractor, and is trained in the same way. The method is used to pre-train the first four layers of a deep convolutional network which achieves state-of-the-art performance on the MNIST dataset of handwritten digits. The final testing error rate is equal to 0.42%. Preliminary experiments on com- pression of bitonal document images show very promising results in terms of compression ratio and reconstruction er- ror.