Statistical analysis of kernel-based least-squares density-ratio estimation

Authors:
Takafumi Kanamori;Taiji Suzuki;Masashi Sugiyama
Affiliations:
Nagoya University, Nagoya, Japan;University of Tokyo, Tokyo, Japan;Tokyo Institute of Technology, Tokyo, Japan
Venue:
Machine Learning
Year:
2012

Citing 0
Cited 7

Importance-weighted least-squares probabilistic classifier for covariate shift adaptation with application to human activity recognition

Neurocomputing
Non-convex optimization on stiefel manifold and applications to machine learning

ICONIP'12 Proceedings of the 19th international conference on Neural Information Processing - Volume Part I
Computational complexity of kernel-based density-ratio estimation: a condition number analysis

Machine Learning
Semi-supervised learning with density-ratio estimation

Machine Learning
Change-point detection in time-series data by relative density-ratio estimation

Neural Networks
Density-difference estimation

Neural Computation
Semi-supervised learning of class balance under class-prior change by distribution matching

Neural Networks

Quantified Score

Hi-index	0.00

Visualization

Abstract

The ratio of two probability densities can be used for solving various machine learning tasks such as covariate shift adaptation (importance sampling), outlier detection (likelihood-ratio test), feature selection (mutual information), and conditional probability estimation. Several methods of directly estimating the density ratio have recently been developed, e.g., moment matching estimation, maximum-likelihood density-ratio estimation, and least-squares density-ratio fitting. In this paper, we propose a kernelized variant of the least-squares method for density-ratio estimation, which is called kernel unconstrained least-squares importance fitting (KuLSIF). We investigate its fundamental statistical properties including a non-parametric convergence rate, an analytic-form solution, and a leave-one-out cross-validation score. We further study its relation to other kernel-based density-ratio estimators. In experiments, we numerically compare various kernel-based density-ratio estimation methods, and show that KuLSIF compares favorably with other approaches.