Reservoir sizes and feedback weights interact non-linearly in echo state networks

Authors:
Danil Koryakin;Martin V. Butz
Affiliations:
Cognitive Modeling, Department of Computer Science, University of Tübingen, Tübingen, Germany;Cognitive Modeling, Department of Computer Science, University of Tübingen, Tübingen, Germany
Venue:
ICANN'12 Proceedings of the 22nd international conference on Artificial Neural Networks and Machine Learning - Volume Part I
Year:
2012

Citing 5
Cited 0

Analysis and design of echo state networks

Neural Computation
2007 Special Issue: An experimental unification of reservoir computing methods

Neural Networks
2007 Special Issue: Decoupled echo state networks with lateral inhibition

Neural Networks
Minimum Complexity Echo State Network

IEEE Transactions on Neural Networks
Balanced echo state networks

Neural Networks

Quantified Score

Hi-index	0.00

Visualization

Abstract

In this paper we investigate parameter dependencies in the echo state network (ESN). In particular, we investigate the interplay between reservoir sizes and the choice of the average absolute output feedback connection weight values (WOFB). We consider the multiple sine wave oscillator problem and the powered sine problem. The results show that somewhat contrary to basic intuition (1) smaller reservoir sizes often yield better networks with higher probability; (2) large WOFB values paired with comparatively large reservoirs may strongly decrease the likelihood of generating effective networks; (3) the likelihood of generating an effective ESN depends non-linearly on the choice of WOFB: very small and large weight values often yield higher likelihoods of generating effective ESNs than networks resulting from intermediate WOFB choices. While the considered test problems are rather simple, the insights gained need to be considered when designing effective ESNs for the problem at hand. Nonetheless, further studies appear necessary to be able to explain the actual reasons behind the observed phenomena.