A Novel Connectionist Network for Solving Long Time-Lag Prediction Tasks

Authors:
Keith Johnson;Cara Macnish
Affiliations:
School of Computer Science and Software Engineering, The University of Western Australia, Australia;School of Computer Science and Software Engineering, The University of Western Australia, Australia
Venue:
AI '09 Proceedings of the 22nd Australasian Joint Conference on Advances in Artificial Intelligence
Year:
2009

Citing 6
Cited 0

A time-delay neural network architecture for isolated word recognition

Neural Networks
The recurrent cascade-correlation architecture

NIPS-3 Proceedings of the 1990 conference on Advances in neural information processing systems 3
Induction of finite-state languages using second-order recurrent networks

Neural Computation
Gradient-based learning algorithms for recurrent networks and their computational complexity

Backpropagation
Learning Sequential Tasks by Incrementally Adding Higher Orders

Advances in Neural Information Processing Systems 5, [NIPS Conference]
Long Short-Term Memory

Neural Computation

Quantified Score

Hi-index	0.00

Visualization

Abstract

Traditional Recurrent Neural Networks (RNNs) perform poorly on learning tasks involving long time-lag dependencies. More recent approaches such as LSTM and its variants significantly improve on RNNs ability to learn this type of problem. We present an alternative approach to encoding temporal dependencies that associates temporal features with nodes rather than state values, where the nodes explicitly encode dependencies over variable time delays. We show promising results comparing the network's performance to LSTM variants on an extended Reber grammar task.