Circular nodes in neural networks

  • Authors:
  • Michael J. Kirby;Rick Miranda

  • Affiliations:
  • Department of Mathematics, Colorado State University, Fort Collins, CO 80523 USA;Department of Mathematics, Colorado State University, Fort Collins, CO 80523 USA

  • Venue:
  • Neural Computation
  • Year:
  • 1996

Quantified Score

Hi-index 0.00

Visualization

Abstract

In the usual construction of a neural network, the individual nodes store and transmit real numbers that lie in an interval on the real line; the values are often envisioned as amplitudes. In this article we present a design for a circular node, which is capable of storing and transmitting angular information. We develop the forward and backward propagation formulas for a network containing circular nodes. We show how the use of circular nodes may facilitate the characterization and parameterization of periodic phenomena in general. We describe applications to constructing circular self-maps, periodic compression, and one-dimensional manifold decomposition. We show that a circular node may be used to construct a homeomorphism between a trefoil knot in â聞聺3 and a unit circle. We give an application with a network that encodes the dynamic system on the limit cycle of the Kuramoto-Sivashinsky equation. This is achieved by incorporating a circular node in the bottleneck layer of a three-hidden-layer bottleneck network architecture. Exploiting circular nodes systematically offers a neural network alternative to Fourier series decomposition in approximating periodic or almost periodic functions.