Dimensionality reduction for density ratio estimation in high-dimensional spaces

Authors:
Masashi Sugiyama;Motoaki Kawanabe;Pui Ling Chui
Affiliations:
Department of Computer Science, Tokyo Institute of Technology, 2-12-1 O-okayama, Meguro-ku, Tokyo 152-8552, Japan and Mathematisches Forchungsinstitute Oberwolfach, Schwarzwaldstr. 9-11, 77709 Obe ...;Fraunhofer Institute FIRST.IDA, Kekuléstr.7, D-12489 Berlin, Germany and Mathematisches Forchungsinstitute Oberwolfach, Schwarzwaldstr. 9-11, 77709 Oberwolfach-Walke, Germany;Department of Computer Science, Tokyo Institute of Technology, 2-12-1 O-okayama, Meguro-ku, Tokyo 152-8552, Japan
Venue:
Neural Networks
Year:
2010

Citing 27
Cited 7

Introduction to statistical pattern recognition (2nd ed.)

Introduction to statistical pattern recognition (2nd ed.)
Elements of information theory

Elements of information theory
Two-sample test statistics for measuring discrepancies between two multivariate probability density functions using kernel-based density estimates

Journal of Multivariate Analysis
Discriminant Adaptive Nearest Neighbor Classification

IEEE Transactions on Pattern Analysis and Machine Intelligence
LOF: identifying density-based local outliers

SIGMOD '00 Proceedings of the 2000 ACM SIGMOD international conference on Management of data
Soft Margins for AdaBoost

Machine Learning
On the influence of the kernel on the consistency of support vector machines

The Journal of Machine Learning Research
Dimensionality Reduction for Supervised Learning with Reproducing Kernel Hilbert Spaces

The Journal of Machine Learning Research
Learning and evaluating classifiers under sample selection bias

ICML '04 Proceedings of the twenty-first international conference on Machine learning
The Entire Regularization Path for the Support Vector Machine

The Journal of Machine Learning Research
Edgeworth Approximation of Multivariate Differential Entropy

Neural Computation
Estimating the Support of a High-Dimensional Distribution

Neural Computation
Integrating structured biological data by Kernel Maximum Mean Discrepancy

Bioinformatics
Active Learning in Approximately Linear Regression Based on Conditional Expectation of Generalization Error

The Journal of Machine Learning Research
Dimensionality Reduction of Multimodal Labeled Data by Local Fisher Discriminant Analysis

The Journal of Machine Learning Research
Discriminative learning for differing training and test distributions

Proceedings of the 24th international conference on Machine learning
An Interior-Point Method for Large-Scale l1-Regularized Logistic Regression

The Journal of Machine Learning Research
Multi-task learning for HIV therapy screening

Proceedings of the 25th international conference on Machine learning
Covariate Shift Adaptation by Importance Weighted Cross Validation

The Journal of Machine Learning Research
Change-Point Detection in Time-Series Data Based on Subspace Identification

ICDM '07 Proceedings of the 2007 Seventh IEEE International Conference on Data Mining
Inlier-Based Outlier Detection via Direct Density Ratio Estimation

ICDM '08 Proceedings of the 2008 Eighth IEEE International Conference on Data Mining
Nonparametric conditional density estimation using piecewise-linear solution path of kernel quantile regression

Neural Computation
Pool-based active learning in approximate linear regression

Machine Learning
Covariate shift adaptation for semi-supervised speaker identification

ICASSP '09 Proceedings of the 2009 IEEE International Conference on Acoustics, Speech and Signal Processing
Adaptive importance sampling for value function approximation in off-policy reinforcement learning

Neural Networks
The use of the area under the ROC curve in the evaluation of machine learning algorithms

Pattern Recognition
On the asymptotic properties of a nonparametric L1-test statistic of homogeneity

IEEE Transactions on Information Theory

Density Ratio Estimation: A New Versatile Tool for Machine Learning

ACML '09 Proceedings of the 1st Asian Conference on Machine Learning: Advances in Machine Learning
Direct density-ratio estimation with dimensionality reduction via least-squares hetero-distributional subspace search

Neural Networks
Least-squares two-sample test

Neural Networks
Importance-weighted least-squares probabilistic classifier for covariate shift adaptation with application to human activity recognition

Neurocomputing
Sequential change-point detection based on direct density-ratio estimation

Statistical Analysis and Data Mining
Change-point detection in time-series data by relative density-ratio estimation

Neural Networks
Density-difference estimation

Neural Computation

Quantified Score

Hi-index	0.00

Visualization

Abstract

The ratio of two probability density functions is becoming a quantity of interest these days in the machine learning and data mining communities since it can be used for various data processing tasks such as non-stationarity adaptation, outlier detection, and feature selection. Recently, several methods have been developed for directly estimating the density ratio without going through density estimation and were shown to work well in various practical problems. However, these methods still perform rather poorly when the dimensionality of the data domain is high. In this paper, we propose to incorporate a dimensionality reduction scheme into a density-ratio estimation procedure and experimentally show that the estimation accuracy in high-dimensional cases can be improved.