Margin based feature selection - theory and algorithms

Authors:
Ran Gilad-Bachrach;Amir Navot;Naftali Tishby
Affiliations:
The Hebrew University, Jerusalem, Israel;The Hebrew University, Jerusalem, Israel;The Hebrew University, Jerusalem, Israel
Venue:
ICML '04 Proceedings of the twenty-first international conference on Machine learning
Year:
2004

Citing 9
Cited 79

A practical approach to feature selection

ML92 Proceedings of the ninth international workshop on Machine learning
Support-Vector Networks

Machine Learning
Self-organizing maps

Self-organizing maps
A decision-theoretic generalization of on-line learning and an application to boosting

Journal of Computer and System Sciences - Special issue: 26th annual ACM symposium on the theory of computing & STOC'94, May 23–25, 1994, and second annual Europe an conference on computational learning theory (EuroCOLT'95), March 13–15, 1995
Wrappers for feature subset selection

Artificial Intelligence - Special issue on relevance
Induction of Decision Trees

Machine Learning
An introduction to variable and feature selection

The Journal of Machine Learning Research
Sufficient dimensionality reduction

The Journal of Machine Learning Research
The sample complexity of pattern classification with neural networks: the size of the weights is more important than the size of the network

IEEE Transactions on Information Theory

OCFS: optimal orthogonal centroid feature selection for text categorization

Proceedings of the 28th annual international ACM SIGIR conference on Research and development in information retrieval
SF-HME system: a hierarchical mixtures-of-experts classification system for spam filtering

Proceedings of the 2006 ACM symposium on Applied computing
Nightmare at test time: robust learning by feature deletion

ICML '06 Proceedings of the 23rd international conference on Machine learning
Iterative RELIEF for feature weighting

ICML '06 Proceedings of the 23rd international conference on Machine learning
Combination of support vector machines using genetic programming

International Journal of Hybrid Intelligent Systems
Using hypothesis margin to boost centroid text classifier

Proceedings of the 2007 ACM symposium on Applied computing
Streamwise Feature Selection

The Journal of Machine Learning Research
A Stochastic Algorithm for Feature Selection in Pattern Recognition

The Journal of Machine Learning Research
Iterative RELIEF for Feature Weighting: Algorithms, Theories, and Applications

IEEE Transactions on Pattern Analysis and Machine Intelligence
Feature selection in a kernel space

Proceedings of the 24th international conference on Machine learning
Ensemble learning for free with evolutionary algorithms?

Proceedings of the 9th annual conference on Genetic and evolutionary computation
Invariant optimal feature selection: A distance discriminant and feature ranking based solution

Pattern Recognition
An efficient feature ranking measure for text categorization

Proceedings of the 2008 ACM symposium on Applied computing
Forward selection method with regression analysis for optimal gene selection in cancer classification

International Journal of Computer Mathematics - Bioinformatics
Heterogeneous data fusion for alzheimer's disease study

Proceedings of the 14th ACM SIGKDD international conference on Knowledge discovery and data mining
Supervised dimensionality reduction via sequential semidefinite programming

Pattern Recognition
A User Independent, Biosignal Based, Emotion Recognition Method

UM '07 Proceedings of the 11th international conference on User Modeling
Applying effective feature selection techniques with hierarchical mixtures of experts for spam classification

Journal of Computer Security
Online Feature Selection Algorithm with Bayesian l 1 Regularization

PAKDD '09 Proceedings of the 13th Pacific-Asia Conference on Advances in Knowledge Discovery and Data Mining
Feature selection based on loss-margin of nearest neighbor classification

Pattern Recognition
Applying effective feature selection techniques with hierarchical mixtures of experts for spam classification

Journal of Computer Security - Best papers of the Sec Track at the 2006 ACM Symposium
Searching for interacting features in subset selection

Intelligent Data Analysis
Method for Prediction of Protein-Protein Interactions in Yeast Using Genomics/Proteomics Information and Feature Selection

IWANN '09 Proceedings of the 10th International Work-Conference on Artificial Neural Networks: Part I: Bio-Inspired Systems: Computational and Ambient Intelligence
Large margin nearest local mean classifier

Signal Processing
Searching for interacting features

IJCAI'07 Proceedings of the 20th international joint conference on Artifical intelligence
Feature selection for fast speech emotion recognition

MM '09 Proceedings of the 17th ACM international conference on Multimedia
Feature Selection via Maximizing Neighborhood Soft Margin

ACML '09 Proceedings of the 1st Asian Conference on Machine Learning: Advances in Machine Learning
Real-time object classification in 3D point clouds using point feature histograms

IROS'09 Proceedings of the 2009 IEEE/RSJ international conference on Intelligent robots and systems
Selecting discrete and continuous features based on neighborhood decision error minimization

IEEE Transactions on Systems, Man, and Cybernetics, Part B: Cybernetics
Tumor classification by combining PNN classifier ensemble with neighborhood rough set based gene reduction

Computers in Biology and Medicine
Gaussian kernel based fuzzy rough sets: Model, uncertainty measures and applications

International Journal of Approximate Reasoning
Neural networks for defect detection in non-destructive evaluation by sonic signals

IWANN'07 Proceedings of the 9th international work conference on Artificial neural networks
New frameworks to boost feature selection algorithms in emotion detection for improved human-computer interaction

BVAI'07 Proceedings of the 2nd international conference on Advances in brain, vision and artificial intelligence
Feature selection by nonparametric Bayes error minimization

PAKDD'08 Proceedings of the 12th Pacific-Asia conference on Advances in knowledge discovery and data mining
Feature evaluation and selection based on neighborhood soft margin

Neurocomputing
Support vector-based feature selection using Fisher's linear discriminant and Support Vector Machine

Expert Systems with Applications: An International Journal
Feature Selection via Maximizing Fuzzy Dependency

Fundamenta Informaticae
Robust relief-feature weighting, margin maximization, and fuzzy optimization

IEEE Transactions on Fuzzy Systems
Soft fuzzy rough sets for robust feature evaluation and selection

Information Sciences: an International Journal
On generating interpretable and precise fuzzy systems based on Pareto multi-objective cooperative co-evolutionary algorithm

Applied Soft Computing
A novel hypothesis-margin based approach for feature selection with side pairwise constraints

Neurocomputing
Weighted nearest neighbor classification via maximizing classification consistency

RSCTC'10 Proceedings of the 7th international conference on Rough sets and current trends in computing
Margin based sample weighting for stable feature selection

WAIM'10 Proceedings of the 11th international conference on Web-age information management
Activities as time series of human postures

ECCV'10 Proceedings of the 11th European conference on Computer vision: Part II
Letters: Large-margin nearest neighbor classifiers via sample weight learning

Neurocomputing
Local margin based semi-supervised discriminant embedding for visual recognition

Neurocomputing
Similarity-margin based feature selection for symbolic interval data

Pattern Recognition Letters
A new dataset evaluation method based on category overlap

Computers in Biology and Medicine
Which clustering do you want? inducing your ideal clustering with minimal feedback

Journal of Artificial Intelligence Research
Dynamic time warping constraint learning for large margin nearest neighbor classification

Information Sciences: an International Journal
Feature level analysis for 3D facial expression recognition

Neurocomputing
Margin-based over-sampling method for learning from imbalanced datasets

PAKDD'11 Proceedings of the 15th Pacific-Asia conference on Advances in knowledge discovery and data mining - Volume Part II
Minimizing neighborhood evidential decision error for feature evaluation and selection based on evidence theory

Expert Systems with Applications: An International Journal
Rule learning for classification based on neighborhood covering reduction

Information Sciences: an International Journal
Driver status recognition by neighborhood covering rules

RSKT'11 Proceedings of the 6th international conference on Rough sets and knowledge technology
Genetic programming for kernel-based learning with co-evolving subsets selection

PPSN'06 Proceedings of the 9th international conference on Parallel Problem Solving from Nature
Using irreducible group representations for invariant 3d shape description

DAGM'06 Proceedings of the 28th conference on Pattern Recognition
Supervised selection of dynamic features, with an application to telecommunication data preparation

ICDM'06 Proceedings of the 6th Industrial Conference on Data Mining conference on Advances in Data Mining: applications in Medicine, Web Mining, Marketing, Image and Signal Mining
An eye-hand data fusion framework for pervasive sensing of surgical activities

Pattern Recognition
Is feature selection still necessary?

SLSFS'05 Proceedings of the 2005 international conference on Subspace, Latent Structure and Feature Selection
Distance-based feature selection on classification of uncertain objects

AI'11 Proceedings of the 24th international conference on Advances in Artificial Intelligence
Large-margin feature selection for monotonic classification

Knowledge-Based Systems
Using machine learning techniques and genomic/proteomic information from known databases for defining relevant features for PPI classification

Computers in Biology and Medicine
A boosting approach for supervised Mahalanobis distance metric learning

Pattern Recognition
A new local search based hybrid genetic algorithm for feature selection

Neurocomputing
Feature weighting by RELIEF based on local hyperplane approximation

PAKDD'12 Proceedings of the 16th Pacific-Asia conference on Advances in Knowledge Discovery and Data Mining - Volume Part II
An effective early fraud detection method for online auctions

Electronic Commerce Research and Applications
Wrapper feature selection for small sample size data driven by complete error estimates

Computer Methods and Programs in Biomedicine
Letters: Dynamic classifier ensemble using classification confidence

Neurocomputing
A variance reduction framework for stable feature selection

Statistical Analysis and Data Mining
Multi-scale patch based collaborative representation for face recognition with margin distribution optimization

ECCV'12 Proceedings of the 12th European conference on Computer Vision - Volume Part I
Discriminative prototype selection methods for graph embedding

Pattern Recognition
RPCA: a novel preprocessing method for PCA

Advances in Artificial Intelligence
Large Margin Subspace Learning for feature selection

Pattern Recognition
On fuzzy-rough attribute selection: Criteria of Max-Dependency, Max-Relevance, Min-Redundancy, and Max-Significance

Applied Soft Computing
Class dependent feature weighting and k-nearest neighbor classification

PRIB'13 Proceedings of the 8th IAPR international conference on Pattern Recognition in Bioinformatics
Robust feature selection based on regularized brownboost loss

Knowledge-Based Systems
Feature selection with test cost constraint

International Journal of Approximate Reasoning
Joint Laplacian feature weights learning

Pattern Recognition

Quantified Score

Hi-index	0.00

Visualization

Abstract

Feature selection is the task of choosing a small set out of a given set of features that capture the relevant properties of the data. In the context of supervised classification problems the relevance is determined by the given labels on the training data. A good choice of features is a key for building compact and accurate classifiers. In this paper we introduce a margin based feature selection criterion and apply it to measure the quality of sets of features. Using margins we devise novel selection algorithms for multi-class classification problems and provide theoretical generalization bound. We also study the well known Relief algorithm and show that it resembles a gradient ascent over our margin criterion. We apply our new algorithm to various datasets and show that our new Simba algorithm, which directly optimizes the margin, outperforms Relief.