Almost optimal lower bounds for small depth circuits
STOC '86 Proceedings of the eighteenth annual ACM symposium on Theory of computing
Parallel distributed processing: explorations in the microstructure of cognition, vol. 2: psychological and biological models
Emergence of grandmother memory in feed forward networks: learning with noise and forgetfulness
Connectionist models and their implications: readings from cognitive science
Connectionist learning procedures
Artificial Intelligence
An application of the principle of maximum information preservation to linear systems
Advances in neural information processing systems 1
Information processing in dynamical systems: foundations of harmony theory
Parallel distributed processing: explorations in the microstructure of cognition, vol. 1
Creating artificial neural networks that generalize
Neural Networks
Training with noise is equivalent to Tikhonov regularization
Neural Computation
Noise injection: theoretical prospects
Neural Computation
Learning continuous attractors in recurrent networks
NIPS '97 Proceedings of the 1997 conference on Advances in neural information processing systems 10
Training products of experts by minimizing contrastive divergence
Neural Computation
Neural Computation
Incorporating Invariances in Support Vector Learning Machines
ICANN 96 Proceedings of the 1996 International Conference on Artificial Neural Networks
Recognition and Structure from one 2D Model View: Observations on Prototypes, Object Classes and Symmetries
Dependency networks for inference, collaborative filtering, and data visualization
The Journal of Machine Learning Research
Nonlinear Autoassociation Is Not Equivalent to PCA
Neural Computation
A fast learning algorithm for deep belief nets
Neural Computation
An empirical evaluation of deep architectures on problems with many factors of variation
Proceedings of the 24th international conference on Machine learning
Backpropagation applied to handwritten zip code recognition
Neural Computation
Extracting and composing robust features with denoising autoencoders
Proceedings of the 25th international conference on Machine learning
Deep learning via semi-supervised embedding
Proceedings of the 25th international conference on Machine learning
Exploring Strategies for Training Deep Neural Networks
The Journal of Machine Learning Research
Justifying and generalizing contrastive divergence
Neural Computation
Learning Deep Architectures for AI
Foundations and Trends® in Machine Learning
Why Does Unsupervised Pre-training Help Deep Learning?
The Journal of Machine Learning Research
Semi-Supervised Learning
A connection between score matching and denoising autoencoders
Neural Computation
Higher order contractive auto-encoder
ECML PKDD'11 Proceedings of the 2011 European conference on Machine learning and knowledge discovery in databases - Volume Part II
On the expressive power of deep architectures
ALT'11 Proceedings of the 22nd international conference on Algorithmic learning theory
Deep networks for predicting human intent with respect to objects
HRI '12 Proceedings of the seventh annual ACM/IEEE international conference on Human-Robot Interaction
Augmented attribute representations
ECCV'12 Proceedings of the 12th European conference on Computer Vision - Volume Part V
Learning invariant feature hierarchies
ECCV'12 Proceedings of the 12th international conference on Computer Vision - Volume Part I
Learning two-layer contractive encodings
ICANN'12 Proceedings of the 22nd international conference on Artificial Neural Networks and Machine Learning - Volume Part I
A system for offline character recognition using auto-encoder networks
ICONIP'12 Proceedings of the 19th international conference on Neural Information Processing - Volume Part IV
Online multimodal deep similarity learning with application to image retrieval
Proceedings of the 21st ACM international conference on Multimedia
Adaptive error-correcting output codes
IJCAI'13 Proceedings of the Twenty-Third international joint conference on Artificial Intelligence
Hi-index | 0.01 |
We explore an original strategy for building deep networks, based on stacking layers of denoising autoencoders which are trained locally to denoise corrupted versions of their inputs. The resulting algorithm is a straightforward variation on the stacking of ordinary autoencoders. It is however shown on a benchmark of classification problems to yield significantly lower classification error, thus bridging the performance gap with deep belief networks (DBN), and in several cases surpassing it. Higher level representations learnt in this purely unsupervised fashion also help boost the performance of subsequent SVM classifiers. Qualitative experiments show that, contrary to ordinary autoencoders, denoising autoencoders are able to learn Gabor-like edge detectors from natural image patches and larger stroke detectors from digit images. This work clearly establishes the value of using a denoising criterion as a tractable unsupervised objective to guide the learning of useful higher level representations.