Minimum description length, regularization, and multimodal data

  • Authors:
  • Richard Rohwer;John C. van der Rest

  • Affiliations:
  • Department of Computer Science and Applied Mathematics, Aston University, Birmingham, B4 7ET, UK;Department of Computer Science and Applied Mathematics, Aston University, Birmingham, B4 7ET, UK

  • Venue:
  • Neural Computation
  • Year:
  • 1996

Quantified Score

Hi-index 0.00

Visualization

Abstract

Relationships between clustering, description length, and regularization are pointed out, motivating the introduction of a cost function with a description length interpretation and the unusual and useful property of having its minimum approximated by the densest mode of a distribution. A simple inverse kinematics example is used to demonstrate that this property can be used to select and learn one branch of a multivalued mapping. This property is also used to develop a method for setting regularization parameters according to the scale on which structure is exhibited in the training data. The regularization technique is demonstrated on two real data sets, a classification problem and a regression problem.