Computational complexity of kernel-based density-ratio estimation: a condition number analysis

  • Authors:
  • Takafumi Kanamori;Taiji Suzuki;Masashi Sugiyama

  • Affiliations:
  • Nagoya University Department of Computer Science and Mathematical Informatics, Nagoya University, Nagoya, Japan 464-8601;Department of Mathematical Informatics, University of Tokyo, Tokyo, Japan 113-8656;Department of Computer Science, Tokyo Institute of Technology, Tokyo, Japan 152-8552

  • Venue:
  • Machine Learning
  • Year:
  • 2013

Quantified Score

Hi-index 0.00

Visualization

Abstract

In this study, the computational properties of a kernel-based least-squares density-ratio estimator are investigated from the viewpoint of condition numbers. The condition number of the Hessian matrix of the loss function is closely related to the convergence rate of optimization and the numerical stability. We use smoothed analysis techniques and theoretically demonstrate that the kernel least-squares method has a smaller condition number than other M-estimators. This implies that the kernel least-squares method has desirable computational properties. In addition, an alternate formulation of the kernel least-squares estimator that possesses an even smaller condition number is presented. The validity of the theoretical analysis is verified through numerical experiments.