On the learning of ESN linear readouts

Authors:
Carlos M. Alaíz;José R. Dorronsoro
Affiliations:
Departamento de Ingeniería Informática, Universidad Autónoma de Madrid, Madrid, Spain;Instituto de Ingeniería del Conocimiento, Universidad Autónoma de Madrid, Madrid, Spain
Venue:
CAEPIA'11 Proceedings of the 14th international conference on Advances in artificial intelligence: spanish association for artificial intelligence
Year:
2011

Citing 7
Cited 0

Adaptive filter theory (3rd ed.)

Adaptive filter theory (3rd ed.)
Real-time computing without stable states: a new framework for neural computation based on perturbations

Neural Computation
Prediction update algorithms for XCSF: RLS, Kalman filter, and gain adaptation

Proceedings of the 8th annual conference on Genetic and evolutionary computation
2007 Special Issue: Edge of chaos and prediction of computational performance for neural circuit models

Neural Networks
A learning algorithm for continually running fully recurrent neural networks

Neural Computation
Convergence analysis of adaptive filtering algorithms with singulardata covariance matrix

IEEE Transactions on Signal Processing
New results on recurrent network training: unifying the algorithms and accelerating convergence

IEEE Transactions on Neural Networks

Quantified Score

Hi-index	0.00

Visualization

Abstract

In the Echo State Networks (ESN) and, more generally, Reservoir Computing paradigms (a recent approach to recurrent neural networks), linear readout weights, i.e., linear output weights, are the only ones actually learned under training. The standard approach for this is SVD-based pseudo-inverse linear regression. Here it will be compared with two well known on-line filters, Least Minimum Squares (LMS) and Recursive Least Squares (RLS). As we shall illustrate, while LMS performance is not satisfactory, RLS can be a good on-line alternative that may deserve further attention.