Regularization and stability in reservoir networks with output feedback

Authors:
René Felix Reinhart;Jochen Jakob Steil
Affiliations:
Research Institute for Cognition and Robotics - CoR-Lab, Bielefeld University, Universitätsstr. 25, 33615 Bielefeld, Germany;Research Institute for Cognition and Robotics - CoR-Lab, Bielefeld University, Universitätsstr. 25, 33615 Bielefeld, Germany
Venue:
Neurocomputing
Year:
2012

Citing 9
Cited 1

Self-organization of distributedly represented multiple behavior schemata in a mirror system: reviews of robot experiments using RNNPB

Neural Networks - 2004 Special issue: New developments in self-organizing systems
On global asymptotic stability of fully connected recurrent neural networks

ICASSP '00 Proceedings of the Acoustics, Speech, and Signal Processing, 2000. on IEEE International Conference - Volume 06
2007 Special Issue: Optimization and applications of echo state networks with leaky- integrator neurons

Neural Networks
Generative Modeling of Autonomous Robots and their Environments using Reservoir Computing

Neural Processing Letters
A smoothing regularizer for feedforward and recurrent neural networks

Neural Computation
Stable Output Feedback in Reservoir Computing Using Ridge Regression

ICANN '08 Proceedings of the 18th international conference on Artificial Neural Networks, Part I
Recurrent Neural Associative Learning of Forward and Inverse Kinematics for Movement Generation of the Redundant PA-10 Robot

LAB-RS '08 Proceedings of the 2008 ECSIS Symposium on Learning and Adaptive Behaviors for Robotic Systems
Online stability of backpropagation-decorrelation recurrent learning

Neurocomputing
A tighter bound for the echo state property

IEEE Transactions on Neural Networks

Multi-directional continuous association with input-driven neural dynamics

Neurocomputing

Quantified Score

Hi-index	0.01

Visualization

Abstract

Output feedback is crucial for autonomous and parameterized pattern generation with reservoir networks. Read-out learning affects the output feedback loop and can lead to error amplification. Regularization is therefore important for both generalization and reduction of error amplification. We show that regularization of the reservoir and the read-out layer reduces the risk of error amplification, mitigates parameter dependency and boosts the task-specific performance of reservoir networks with output feedback. We discuss the deeper connection between regularization of the learning process and stability of the trained network.