Divergence, Optimization and Geometry

  • Authors:
  • Shun-Ichi Amari

  • Affiliations:
  • RIKEN Brain Science Institute, Saitama, Japan 351-0198

  • Venue:
  • ICONIP '09 Proceedings of the 16th International Conference on Neural Information Processing: Part I
  • Year:
  • 2009

Quantified Score

Hi-index 0.00

Visualization

Abstract

Measures of divergence are used in many engineering problems such as statistics, mathematical programming, computational vision, and neural networks. The Kullback-Leibler divergence is its typical example which is defined between two probability distributions, and is invariant under information transformations. The Bregman divergence is another type of divergence, which are used often in optimization and signal processing. This is a class of divergences having dually flat geometrical structure. Divergence is often used for minimizing discrepancy between observed evidences and an underlying model. Projection to the model subspace plays a fundamental role. Here, geometry is important and dually flat geodesic structure is useful, because a generalized Pythagorean theorem and projection theorem hold.