Training recurrent connectionist models on symbolic time series

Authors:
Michal Čerňanský;Ľubica Beňušková
Affiliations:
Faculty of Informatics and Information Technologies, STU Bratislava, Slovakia;Department of Computer Science, University of Otago, Dunedin, New Zealand
Venue:
ICONIP'08 Proceedings of the 15th international conference on Advances in neuro-information processing - Volume Part I
Year:
2008

Citing 7
Cited 0

Distributed Representations, Simple Recurrent Networks, And Grammatical Structure

Machine Learning - Connectionist approaches to language learning
Predicting the Future of Discrete Sequences from Fractal Representations of the Past

Machine Learning
Simple Recurrent Networks Learn Context-Free and Context-Sensitive Languages by Counting

Neural Computation
Long Short-Term Memory

Neural Computation
2007 Special Issue: Learning grammatical structure with Echo State Networks

Neural Networks
On learning context-free and context-sensitive languages

IEEE Transactions on Neural Networks
Markovian architectural bias of recurrent neural networks

IEEE Transactions on Neural Networks

Quantified Score

Hi-index	0.01

Visualization

Abstract

This work provide a short study of training algorithms useful for adaptation of recurrent connectionist models for symbolic time series modeling tasks. We show that approaches based on Kalman filtration outperform standard gradinet based training algorithms. We propose simple approximation to the Kalman filtration with favorable computational requirements and on several linguistic time series taken from recently published papers we demonstrate superior ability of the proposed method.