Improving the Learning Speed in 2-Layered LSTM Network by Estimating the Configuration of Hidden Units and Optimizing Weights Initialization

Authors:
Débora C. Corrêa;Alexandre L. Levada;José H. Saito
Affiliations:
Computer Department, Federal University of São Carlos, São Paulo, Brazil;Physics Institute of São Carlos, University of São Paulo, São Paulo, Brazil;Computer Department, Federal University of São Carlos, São Paulo, Brazil
Venue:
ICANN '08 Proceedings of the 18th international conference on Artificial Neural Networks, Part I
Year:
2008

Citing 9
Cited 0

Gradient-based learning algorithms for recurrent networks and their computational complexity

Backpropagation
Principles of computerized tomographic imaging

Principles of computerized tomographic imaging
Neural Networks: A Comprehensive Foundation

Neural Networks: A Comprehensive Foundation
Shape Analysis and Classification: Theory and Practice

Shape Analysis and Classification: Theory and Practice
Kalman filters improve LSTM network performance in problems unsolvable by traditional recurrent nets

Neural Networks
2005 Special Issue: Framewise phoneme classification with bidirectional LSTM and other neural network architectures

Neural Networks - 2005 Special issue: IJCNN 2005
Recurrent Neural Networks for Music Computation

INFORMS Journal on Computing
Long Short-Term Memory

Neural Computation
Training Recurrent Networks by Evolino

Neural Computation

Quantified Score

Hi-index	0.00

Visualization

Abstract

This paper describes a method to initialize the LSTM network weights and estimate the configuration of hidden units in order to improve training time for function approximation tasks. The motivation of this method is based on the behavior of the hidden units and the complexity of the function to be approximated. The results obtained for 1-D and 2-D functions show that the proposed methodology improves the network performance, stabilizing the training phase.