Self-organized reservoirs and their hierarchies

Authors:
Mantas Lukoševičius
Affiliations:
Jacobs University Bremen, Bremen, Germany
Venue:
ICANN'12 Proceedings of the 22nd international conference on Artificial Neural Networks and Machine Learning - Volume Part I
Year:
2012

Citing 9
Cited 0

The temporal Kohonen map

Neural Networks
A Recurrent Self-Organizing Map for Temporal Sequence Processing

ICANN '97 Proceedings of the 7th International Conference on Artificial Neural Networks
Recursive self-organizing maps

Neural Networks - New developments in self-organizing maps
Recursive self-organizing network models

Neural Networks - 2004 Special issue: New developments in self-organizing systems
Dynamics and Topographic Organization of Recursive Self-Organizing Maps

Neural Computation
2007 Special Issue: Optimization and applications of echo state networks with leaky- integrator neurons

Neural Networks
Backpropagation applied to handwritten zip code recognition

Neural Computation
Why Does Unsupervised Pre-training Help Deep Learning?

The Journal of Machine Learning Research
Survey: Reservoir computing approaches to recurrent neural network training

Computer Science Review

Quantified Score

Hi-index	0.00

Visualization

Abstract

We investigate how unsupervised training of recurrent neural networks (RNNs) and their deep hierarchies can benefit a supervised task like temporal pattern detection. The RNNs are fully and fast trained by unsupervised algorithms and only supervised feed-forward readouts are used. The unsupervised RNNs are shown to perform better in a rigorous comparison against state-of-art random reservoir networks. Unsupervised greedy bottom-up trained hierarchies of such RNNs are shown being capable of big performance improvements over single layer setups.