The expressive power of analog recurrent neural networks on infinite input streams

  • Authors:

  • Jérémie Cabessa;Alessandro E. P. Villa

  • Affiliations:

  • Grenoble Institute of Neuroscience, Inserm UMRS 836, University Joseph Fourier, FR-38041 Grenoble, France;Department of Information Systems, Faculty of Business and Economics, University of Lausanne, CH-1015 Lausanne, Switzerland

  • Venue:
  • Theoretical Computer Science
  • Year:
  • 2012

Quantified Score

Hi-index 5.23

Visualization

Abstract

We consider analog recurrent neural networks working on infinite input streams, provide a complete topological characterization of their expressive power, and compare it to the expressive power of classical infinite word reading abstract machines. More precisely, we consider analog recurrent neural networks as language recognizers over the Cantor space, and prove that the classes of @w-languages recognized by deterministic and non-deterministic analog networks correspond precisely to the respective classes of @P"2^0-sets and @S"1^1-sets of the Cantor space. Furthermore, we show that the result can be generalized to more expressive analog networks equipped with any kind of Borel accepting condition. Therefore, in the deterministic case, the expressive power of analog neural nets turns out to be comparable to the expressive power of any kind of Buchi abstract machine, whereas in the non-deterministic case, analog recurrent networks turn out to be strictly more expressive than any other kind of Buchi or Muller abstract machine, including the main cases of classical automata, 1-counter automata, k-counter automata, pushdown automata, and Turing machines.