Direct density-ratio estimation with dimensionality reduction via least-squares hetero-distributional subspace search

Authors:
Masashi Sugiyama;Makoto Yamada;Paul von Bünau;Taiji Suzuki;Takafumi Kanamori;Motoaki Kawanabe
Affiliations:
Tokyo Institute of Technology, 2-12-1 O-okayama, Meguro-ku, Tokyo 152-8552, Japan and PRESTO, Japan Science and Technology Agency, Japan;Tokyo Institute of Technology, 2-12-1 O-okayama, Meguro-ku, Tokyo 152-8552, Japan;Technical University of Berlin Franklinstr. 28/29, 10587 Berlin, Germany;The University of Tokyo, 7-3-1 Hongo, Bunkyo-ku, Tokyo 113-8656, Japan;Nagoya University, Furocho, Chikusaku, Nagoya 464-8603, Japan;Fraunhofer FIRST.IDA, Kekuléstr. 7, 12489 Berlin, Germany
Venue:
Neural Networks
Year:
2011

Citing 27
Cited 7

Natural gradient works efficiently in learning

Neural Computation
On the influence of the kernel on the consistency of support vector machines

The Journal of Machine Learning Research
Learning and evaluating classifiers under sample selection bias

ICML '04 Proceedings of the twenty-first international conference on Machine learning
Edgeworth Approximation of Multivariate Differential Entropy

Neural Computation
Active Learning in Approximately Linear Regression Based on Conditional Expectation of Generalization Error

The Journal of Machine Learning Research
Dimensionality Reduction of Multimodal Labeled Data by Local Fisher Discriminant Analysis

The Journal of Machine Learning Research
Discriminative learning for differing training and test distributions

Proceedings of the 24th international conference on Machine learning
Multi-task learning for HIV therapy screening

Proceedings of the 25th international conference on Machine learning
Covariate Shift Adaptation by Importance Weighted Cross Validation

The Journal of Machine Learning Research
Inlier-Based Outlier Detection via Direct Density Ratio Estimation

ICDM '08 Proceedings of the 2008 Eighth IEEE International Conference on Data Mining
Nonparametric conditional density estimation using piecewise-linear solution path of kernel quantile regression

Neural Computation
Estimating Squared-Loss Mutual Information for Independent Component Analysis

ICA '09 Proceedings of the 8th International Conference on Independent Component Analysis and Signal Separation
Pool-based active learning in approximate linear regression

Machine Learning
Efficient Sample Reuse in EM-Based Policy Search

ECML PKDD '09 Proceedings of the European Conference on Machine Learning and Knowledge Discovery in Databases: Part I
Dimensionality reduction for density ratio estimation in high-dimensional spaces

Neural Networks
Adaptive importance sampling for value function approximation in off-policy reinforcement learning

Neural Networks
Mutual information approximation via maximum likelihood estimation of density ratio

ISIT'09 Proceedings of the 2009 IEEE international conference on Symposium on Information Theory - Volume 1
Learning algorithms utilizing quasi-geodesic flows on the Stiefel manifold

Neurocomputing
Geometrical methods for non-negative ICA: Manifolds, Lie groups and toral subalgebras

Neurocomputing
The use of the area under the ROC curve in the evaluation of machine learning algorithms

Pattern Recognition
A Least-squares Approach to Direct Importance Estimation

The Journal of Machine Learning Research
Efficient exploration through active learning for value function approximation in reinforcement learning

Neural Networks
Semi-supervised speaker identification under covariate shift

Signal Processing
Perceived Age Estimation under Lighting Condition Change by Covariate Shift Adaptation

ICPR '10 Proceedings of the 2010 20th International Conference on Pattern Recognition
Estimating divergence functionals and the likelihood ratio by convex risk minimization

IEEE Transactions on Information Theory
Statistical outlier detection using direct density ratio estimation

Knowledge and Information Systems
Density Ratio Estimation in Machine Learning

Density Ratio Estimation in Machine Learning

Least-squares two-sample test

Neural Networks
Importance-weighted least-squares probabilistic classifier for covariate shift adaptation with application to human activity recognition

Neurocomputing
Sequential change-point detection based on direct density-ratio estimation

Statistical Analysis and Data Mining
Detecting interesting events using unsupervised density ratio estimation

ECCV'12 Proceedings of the 12th international conference on Computer Vision - Volume Part III
Non-convex optimization on stiefel manifold and applications to machine learning

ICONIP'12 Proceedings of the 19th international conference on Neural Information Processing - Volume Part I
Change-point detection in time-series data by relative density-ratio estimation

Neural Networks
Density-difference estimation

Neural Computation

Quantified Score

Hi-index	0.00

Visualization

Abstract

Methods for directly estimating the ratio of two probability density functions have been actively explored recently since they can be used for various data processing tasks such as non-stationarity adaptation, outlier detection, and feature selection. In this paper, we develop a new method which incorporates dimensionality reduction into a direct density-ratio estimation procedure. Our key idea is to find a low-dimensional subspace in which densities are significantly different and perform density-ratio estimation only in this subspace. The proposed method, D^3-LHSS (Direct Density-ratio estimation with Dimensionality reduction via Least-squares Hetero-distributional Subspace Search), is shown to overcome the limitation of baseline methods.