Structural Properties of Recurrent Neural Networks

Authors:
Andrej Dobnikar;Branko Šter
Affiliations:
Faculty of Computer and Information Science, University of Ljubljana, Ljubljana, Slovenia 1000;Faculty of Computer and Information Science, University of Ljubljana, Ljubljana, Slovenia 1000
Venue:
Neural Processing Letters
Year:
2009

Citing 7
Cited 1

An algorithm for drawing general undirected graphs

Information Processing Letters
Grammatical Inference using an Adaptive Recurrent Neural Network

Neural Processing Letters
Handbook of Graphs and Networks: From the Genome to the Internet

Handbook of Graphs and Networks: From the Genome to the Internet
On-line identification and reconstruction of finite automata with generalized recurrent neural networks

Neural Networks
Finite state automata and simple recurrent networks

Neural Computation
A learning algorithm for continually running fully recurrent neural networks

Neural Computation
Spontaneous scale-free structure of spike flow graphs in recurrent neural networks

Neural Networks

Scalability of learning impact on complex parameters in recurrent neural networks

ICANNGA'09 Proceedings of the 9th international conference on Adaptive and natural computing algorithms

Quantified Score

Hi-index	0.00

Visualization

Abstract

In this article we research the impact of the adaptive learning process of recurrent neural networks (RNN) on the structural properties of the derived graphs. A trained fully connected RNN can be converted to a graph by defining edges between pairs od nodes having significant weights. We measured structural properties of the derived graphs, such as characteristic path lengths, clustering coefficients and degree distributions. The results imply that a trained RNN has significantly larger clustering coefficient than a random network with a comparable connectivity. Besides, the degree distributions show existence of nodes with a large degree or hubs, typical for scale-free networks. We also show analytically and experimentally that this type of degree distribution has increased entropy.