Pruning and regularization in reservoir computing

Authors:
X. Dutoit;B. Schrauwen;J. Van Campenhout;D. Stroobandt;H. Van Brussel;M. Nuttin
Affiliations:
Group of Mobile Learning Robots, Division of Production Engineering, Machine Design and Automation, Department of Mechanical Engineering, Katholieke Universiteit Leuven, Celestijnenlaan 300B, 3001 ...;Group of Parallel Information Systems, Department of Electronics and Information Systems, Universiteit Gent, Sint Pietersnieuwstraat 41, 9000 Gent, Belgium;Group of Parallel Information Systems, Department of Electronics and Information Systems, Universiteit Gent, Sint Pietersnieuwstraat 41, 9000 Gent, Belgium;Group of Parallel Information Systems, Department of Electronics and Information Systems, Universiteit Gent, Sint Pietersnieuwstraat 41, 9000 Gent, Belgium;Group of Mobile Learning Robots, Division of Production Engineering, Machine Design and Automation, Department of Mechanical Engineering, Katholieke Universiteit Leuven, Celestijnenlaan 300B, 3001 ...;Group of Mobile Learning Robots, Division of Production Engineering, Machine Design and Automation, Department of Mechanical Engineering, Katholieke Universiteit Leuven, Celestijnenlaan 300B, 3001 ...
Venue:
Neurocomputing
Year:
2009

Citing 11
Cited 5

Ridge regression: biased estimation for nonorthogonal problems

Technometrics
Real-time computing without stable states: a new framework for neural computation based on perturbations

Neural Computation
Isolated word recognition with the liquid state machine: a case study

Information Processing Letters - Special issue on applications of spiking neural networks
Introduction to Linear Regression Analysis, Solutions Manual (Wiley Series in Probability and Statistics)

Introduction to Linear Regression Analysis, Solutions Manual (Wiley Series in Probability and Statistics)
Training Recurrent Networks by Evolino

Neural Computation
2007 Special Issue: Edge of chaos and prediction of computational performance for neural circuit models

Neural Networks
2007 Special Issue: Optimization and applications of echo state networks with leaky- integrator neurons

Neural Networks
2007 Special Issue: Decoupled echo state networks with lateral inhibition

Neural Networks
Deterministic neural classification

Neural Computation
Event detection and localization in mobile robot navigation using reservoir computing

ICANN'07 Proceedings of the 17th international conference on Artificial neural networks
New results on recurrent network training: unifying the algorithms and accelerating convergence

IEEE Transactions on Neural Networks

A comparative study of Reservoir Computing strategies for monthly time series prediction

Neurocomputing
Echo state networks with sparse output connections

ICANN'10 Proceedings of the 20th international conference on Artificial neural networks: Part I
Simple deterministically constructed cycle reservoirs with regular jumps

Neural Computation
Modular state space of echo state network

Neurocomputing
Optimized Parameter Search for Large Datasets of the Regularization Parameter and Feature Selection for Ridge Regression

Neural Processing Letters

Quantified Score

Hi-index	0.03

Visualization

Abstract

Reservoir computing is a new paradigm for using recurrent neural network with a much simpler training method. The key idea is to use a large but fixed recurrent part as a reservoir of dynamic features and to train only the output layer to extract the desired information. We propose to study how pruning some connections from the reservoir to the output layer can help on the one hand to increase the generalization ability, in much the same way as regularization techniques do, and on the other hand to improve the implementability of reservoirs in hardware.