Survey: Reservoir computing approaches to recurrent neural network training

  • Authors:
  • Mantas Lukoševičius;Herbert Jaeger

  • Affiliations:
  • School of Engineering and Science, Jacobs University Bremen gGmbH, P.O. Box 750 561, 28725 Bremen, Germany;School of Engineering and Science, Jacobs University Bremen gGmbH, P.O. Box 750 561, 28725 Bremen, Germany

  • Venue:
  • Computer Science Review
  • Year:
  • 2009

Quantified Score

Hi-index 0.00

Visualization

Abstract

Echo State Networks and Liquid State Machines introduced a new paradigm in artificial recurrent neural network (RNN) training, where an RNN (the reservoir) is generated randomly and only a readout is trained. The paradigm, becoming known as reservoir computing, greatly facilitated the practical application of RNNs and outperformed classical fully trained RNNs in many tasks. It has lately become a vivid research field with numerous extensions of the basic idea, including reservoir adaptation, thus broadening the initial paradigm to using different methods for training the reservoir and the readout. This review systematically surveys both current ways of generating/adapting the reservoirs and training different types of readouts. It offers a natural conceptual classification of the techniques, which transcends boundaries of the current ''brand-names'' of reservoir methods, and thus aims to help in unifying the field and providing the reader with a detailed ''map'' of it.