Adaptive learning of linguistic hierarchy in a multiple timescale recurrent neural network

Authors:
Stefan Heinrich;Cornelius Weber;Stefan Wermter
Affiliations:
Department of Informatics, Knowledge Technology, University of Hamburg, Hamburg, Germany;Department of Informatics, Knowledge Technology, University of Hamburg, Hamburg, Germany;Department of Informatics, Knowledge Technology, University of Hamburg, Hamburg, Germany
Venue:
ICANN'12 Proceedings of the 22nd international conference on Artificial Neural Networks and Machine Learning - Volume Part I
Year:
2012

Citing 6
Cited 0

Learning internal representations by error propagation

Parallel distributed processing: explorations in the microstructure of cognition, vol. 1
Gradient-based learning algorithms for recurrent networks and their computational complexity

Backpropagation
Hybrid neural plausibility networks for news agents

AAAI '99/IAAI '99 Proceedings of the sixteenth national conference on Artificial intelligence and the eleventh Innovative applications of artificial intelligence conference innovative applications of artificial intelligence
Field Guide to Dynamical Recurrent Networks

Field Guide to Dynamical Recurrent Networks
Emergence of hierarchical structure mirroring linguistic composition in a recurrent neural network

Neural Networks
Self-organization of behavioral primitives as multiple attractor dynamics: A robot experiment

IEEE Transactions on Systems, Man, and Cybernetics, Part A: Systems and Humans

Quantified Score

Hi-index	0.00

Visualization

Abstract

Recent research has revealed that hierarchical linguistic structures can emerge in a recurrent neural network with a sufficient number of delayed context layers. As a representative of this type of network the Multiple Timescale Recurrent Neural Network (MTRNN) has been proposed for recognising and generating known as well as unknown linguistic utterances. However the training of utterances performed in other approaches demands a high training effort. In this paper we propose a robust mechanism for adaptive learning rates and internal states to speed up the training process substantially. In addition we compare the generalisation of the network for the adaptive mechanism as well as the standard fixed learning rates finding at least equal capabilities.