Feature selection via dependence maximization

Authors:
Le Song;Alex Smola;Arthur Gretton;Justin Bedo;Karsten Borgwardt
Affiliations:
Computational Science and Engineering, Georgia Institute of Technology, Atlanta, GA;Yahoo! Research, Santa Clara, CA;Gatsby Computational Neuroscience Unit, London, UK and Intelligent Systems Group, Max Planck Institutes, Tübingen, Germany;Statistical Machine Learning Program, National ICT Australia, Canberra, ACT, Australia and Australian National University, Canberra, ACT, Australia;Machine Learning and Computational Biology Research Group, Max Planck Institutes, Tübingen, Germany
Venue:
The Journal of Machine Learning Research
Year:
2012

Citing 23
Cited 2

A practical approach to feature selection

ML92 Proceedings of the ninth international workshop on Machine learning
Two-sample test statistics for measuring discrepancies between two multivariate probability density functions using kernel-based density estimates

Journal of Multivariate Analysis
Shrinking the tube: a new support vector regression algorithm

Proceedings of the 1998 conference on Advances in neural information processing systems II
Gene Selection for Cancer Classification using Support Vector Machines

Machine Learning
Feature Selection via Concave Minimization and Support Vector Machines

ICML '98 Proceedings of the Fifteenth International Conference on Machine Learning
The concave-convex procedure

Neural Computation
On the influence of the kernel on the consistency of support vector machines

The Journal of Machine Learning Research
Text classification using string kernels

The Journal of Machine Learning Research
Kernel independent component analysis

The Journal of Machine Learning Research
Minimal kernel classifiers

The Journal of Machine Learning Research
An introduction to variable and feature selection

The Journal of Machine Learning Research
Use of the zero norm with linear models and kernel methods

The Journal of Machine Learning Research
Dimensionality Reduction for Supervised Learning with Reproducing Kernel Hilbert Spaces

The Journal of Machine Learning Research
Near-optimal sensor placements in Gaussian processes

ICML '05 Proceedings of the 22nd international conference on Machine learning
Combined SVM-Based Feature Selection and Classification

Machine Learning
Binet-Cauchy Kernels on Dynamical Systems and its Application to the Analysis of Dynamic Scenes

International Journal of Computer Vision
Supervised feature selection via dependence estimation

Proceedings of the 24th international conference on Machine learning
Gene selection via the BAHSIC family of algorithms

Bioinformatics
A kernel approach to comparing distributions

AAAI'07 Proceedings of the 22nd national conference on Artificial intelligence - Volume 2
Hilbert Space Embeddings and Metrics on Probability Measures

The Journal of Machine Learning Research
Robust feature selection by mutual information distributions

UAI'02 Proceedings of the Eighteenth conference on Uncertainty in artificial intelligence
Measuring statistical dependence with hilbert-schmidt norms

ALT'05 Proceedings of the 16th international conference on Algorithmic Learning Theory
An efficient alternative to SVM based recursive feature elimination with applications in natural language processing and bioinformatics

AI'06 Proceedings of the 19th Australian joint conference on Artificial Intelligence: advances in Artificial Intelligence

Spatial distance join based feature selection

Engineering Applications of Artificial Intelligence
Alternative second-order cone programming formulations for support vector classification

Information Sciences: an International Journal

Quantified Score

Hi-index	0.00

Visualization

Abstract

We introduce a framework for feature selection based on dependence maximization between the selected features and the labels of an estimation problem, using the Hilbert-Schmidt Independence Criterion. The key idea is that good features should be highly dependent on the labels. Our approach leads to a greedy procedure for feature selection. We show that a number of existing feature selectors are special cases of this framework. Experiments on both artificial and real-world data show that our feature selector works well in practice.