Simple Strategies to Encode Tree Automata in Sigmoid Recursive Neural Networks

Authors:
Rafael C. Carrasco;Mikel L. Forcada
Affiliations:
-;-
Venue:
IEEE Transactions on Knowledge and Data Engineering
Year:
2001

Citing 12
Cited 6

Formal languages

Formal languages
Learning and extracting finite state automata with second-order recurrent neural networks

Neural Computation
Learning the initial state of a second-order recurrent neural network during regular-language inference

Neural Computation
An algebraic framework to represent finite state machines in single-layer recurrent neural networks

Neural Computation
Constructing deterministic finite-state automata in recurrent neural networks

Journal of the ACM (JACM)
On the computational power of recurrent neural networks for structures

Neural Networks
Introduction To Automata Theory, Languages, And Computation

Introduction To Automata Theory, Languages, And Computation
Stable Encoding of Finite-State Machines in Discrete-Time Recurrent Neural Nets with Sigmoid Units

Neural Computation
Stable encoding of large finite-state automata in recurrent neural networks with sigmoid discriminants

Neural Computation
Supervised neural networks for the classification of structures

IEEE Transactions on Neural Networks
A general framework for adaptive processing of data structures

IEEE Transactions on Neural Networks
On the computational power of Elman-style recurrent networks

IEEE Transactions on Neural Networks

Encoding Nondeterministic Finite-State Tree Automata in Sigmoid Recursive Neural Networks

Proceedings of the Joint IAPR International Workshops on Advances in Pattern Recognition
Neural Networks for Adaptive Processing of Structured Data

ICANN '01 Proceedings of the International Conference on Artificial Neural Networks
Recurrent neural networks with small weights implement definite memory machines

Neural Computation
Recursive self-organizing network models

Neural Networks - 2004 Special issue: New developments in self-organizing systems
Rule Extraction from Recurrent Neural Networks: A Taxonomy and Review

Neural Computation
Recurrent networks for structured data - A unifying approach and its properties

Cognitive Systems Research

Quantified Score

Hi-index	0.00

Visualization

Abstract

Recently, a number of authors have explored the use of recursive neural nets (RNN) for the adaptive processing of trees or tree-like structures. One of the most important language-theoretical formalizations of the processing of tree-structured data is that of deterministic finite-state tree automata (DFSTA). DFSTA may easily be realized as RNN using discrete-state units, such as the threshold linear unit. A recent result by Síima (Neural Network World7 (1997), pp. 679驴686) shows that any threshold linear unit operating on binary inputs can be implemented in an analog unit using a continuous activation function and bounded real inputs. The constructive proof finds a scaling factor for the weights and reestimates the bias accordingly. In this paper, we explore the application of this result to simulate DFSTA in sigmoid RNN (that is, analog RNN using monotonically growing activation functions) and also present an alternative scheme for one-hot encoding of the input that yields smaller weight values and, therefore, works at a lower saturation level.