A Characterization of Simple Recurrent Neural Networks with Two Hidden Units as a Language Recognizer

Authors:
Azusa Iwata;Yoshihisa Shinozawa;Akito Sakurai
Affiliations:
Keio University, Hiyoshi, Kohoku-ku, Japan 223-8522;Keio University, Hiyoshi, Kohoku-ku, Japan 223-8522;Keio University, Hiyoshi, Kohoku-ku, Japan 223-8522 and CREST, Japan Science and Technology Agency,
Venue:
Neural Information Processing
Year:
2007

Citing 11
Cited 0

Distributed Representations, Simple Recurrent Networks, And Grammatical Structure

Machine Learning - Connectionist approaches to language learning
Language as a dynamical system

Mind as motion
Correction to proof that recurrent neural networks can robustly recognize only regular languages

Neural Computation
On the effect of analog noise in discrete-time analog computations

Neural Computation
Neural networks and analog computation: beyond the Turing limit

Neural networks and analog computation: beyond the Turing limit
Introduction To Automata Theory, Languages, And Computation

Introduction To Automata Theory, Languages, And Computation
Learning nonregular languages: a comparison of simple recurrent networks and LSTM

Neural Computation
Learning the Dynamics of Embedded Clauses

Applied Intelligence
Incremental training of first order recurrent neural networks to predict a context-sensitive language

Neural Networks
Simple Recurrent Networks Learn Context-Free and Context-Sensitive Languages by Counting

Neural Computation
LSTM recurrent networks learn simple context-free and context-sensitive languages

IEEE Transactions on Neural Networks

Quantified Score

Hi-index	0.00

Visualization

Abstract

We give a necessary condition that a simple recurrent neural network with two sigmoidal hidden units to implement a recognizer of the formal language {anbn| n 0 } which is generated by a set of generating rules {S茂戮驴aSb, S茂戮驴ab} and show that by setting parameters so as to conform to the condition we get a recognizer of the language. The condition implies instability of learning process reported in previous studies. The condition also implies, contrary to its success in implementing the recognizer, difficulty of getting a recognizer of more complicated languages.