Selective Recurrent Neural Network

Authors:
Branko Šter
Affiliations:
Faculty of Computer and Information Science, University of Ljubljana, Ljubljana, Slovenia 1000
Venue:
Neural Processing Letters
Year:
2013

Citing 11
Cited 0

Learning complex, extended sequences using the principle of history compression

Neural Computation
Grammatical Inference using an Adaptive Recurrent Neural Network

Neural Processing Letters
Kalman Filtering and Neural Networks

Kalman Filtering and Neural Networks
On-line identification and reconstruction of finite automata with generalized recurrent neural networks

Neural Networks
Learning to Forget: Continual Prediction with LSTM

Neural Computation
Long Short-Term Memory

Neural Computation
Finite state automata and simple recurrent networks

Neural Computation
A learning algorithm for continually running fully recurrent neural networks

Neural Computation
Learning long term dependencies with recurrent neural networks

ICANN'06 Proceedings of the 16th international conference on Artificial Neural Networks - Volume Part I
Learning long-term dependencies in NARX recurrent neural networks

IEEE Transactions on Neural Networks
Learning long-term dependencies with gradient descent is difficult

IEEE Transactions on Neural Networks

Quantified Score

Hi-index	0.00

Visualization

Abstract

It is known that recurrent neural networks may have difficulties remembering data over long time lags. To overcome this problem, we propose an extended architecture of recurrent neural networks, which is able to deal with long time lags between relevant input signals. A register of latches at the input layer of the network is applied to bypass irrelevant input information and to propagate relevant inputs. The latches are implemented with differentiable multiplexers, thus enabling the derivatives to be propagated through the network. The relevance of input vectors is learned concurrently with the weights of the network using a gradient-based algorithm.