Computational power of neural networks: a characterization in terms of Kolmogorov complexity

  • Authors:
  • J. L. Balcazar;R. Gavalda;H. T. Siegelmann

  • Affiliations:
  • Dept. of Software, Univ. Politecnica de Catalunya, Barcelona;-;-

  • Venue:
  • IEEE Transactions on Information Theory
  • Year:
  • 2006

Quantified Score

Hi-index 754.84

Visualization

Abstract

The computational power of recurrent neural networks is shown to depend ultimately on the complexity of the real constants (weights) of the network. The complexity, or information contents, of the weights is measured by a variant of resource-bounded Kolmogorov (1965) complexity, taking into account the time required for constructing the numbers. In particular, we reveal a full and proper hierarchy of nonuniform complexity classes associated with networks having weights of increasing Kolmogorov complexity