Divergence-based feature selection for separate classes

Authors:
Yishi Zhang;Shujuan Li;Teng Wang;Zigang Zhang
Affiliations:
School of Management, Huazhong University of Science and Technology, 1037 Luoyu Road, Hongshan District, Wuhan 430074, China;School of Management, Huazhong University of Science and Technology, 1037 Luoyu Road, Hongshan District, Wuhan 430074, China;School of Software Engineering, Huazhong University of Science and Technology, Wuhan 430074, China;School of Management, Huazhong University of Science and Technology, 1037 Luoyu Road, Hongshan District, Wuhan 430074, China
Venue:
Neurocomputing
Year:
2013

Citing 30
Cited 1

Probabilistic reasoning in intelligent systems: networks of plausible inference

Probabilistic reasoning in intelligent systems: networks of plausible inference
Instance-Based Learning Algorithms

Machine Learning
A practical approach to feature selection

ML92 Proceedings of the ninth international workshop on Machine learning
C4.5: programs for machine learning

C4.5: programs for machine learning
Estimating attributes: analysis and extensions of RELIEF

ECML-94 Proceedings of the European conference on machine learning on Machine Learning
Data mining: practical machine learning tools and techniques with Java implementations

Data mining: practical machine learning tools and techniques with Java implementations
An introduction to support Vector Machines: and other kernel-based learning methods

An introduction to support Vector Machines: and other kernel-based learning methods
Correlation-based Feature Selection for Discrete and Numeric Class Machine Learning

ICML '00 Proceedings of the Seventeenth International Conference on Machine Learning
Artificial Intelligence: A Modern Approach

Artificial Intelligence: A Modern Approach
Minimum Redundancy Feature Selection from Microarray Gene Expression Data

CSB '03 Proceedings of the IEEE Computer Society Conference on Bioinformatics
Theoretical and Empirical Analysis of ReliefF and RReliefF

Machine Learning
An introduction to variable and feature selection

The Journal of Machine Learning Research
Feature selection with conditional mutual information maximin in text categorization

Proceedings of the thirteenth ACM international conference on Information and knowledge management
Efficient Feature Selection via Analysis of Relevance and Redundancy

The Journal of Machine Learning Research
Fast Binary Feature Selection with Conditional Mutual Information

The Journal of Machine Learning Research
Toward Integrating Feature Selection Algorithms for Classification and Clustering

IEEE Transactions on Knowledge and Data Engineering
Feature Selection Based on Mutual Information: Criteria of Max-Dependency, Max-Relevance, and Min-Redundancy

IEEE Transactions on Pattern Analysis and Machine Intelligence
A New Dependency and Correlation Analysis for Features

IEEE Transactions on Knowledge and Data Engineering
Speculative Markov Blanket Discovery for Optimal Feature Selection

ICDM '05 Proceedings of the Fifth IEEE International Conference on Data Mining
The max-min hill-climbing Bayesian network structure learning algorithm

Machine Learning
Towards scalable and data efficient learning of Markov boundaries

International Journal of Approximate Reasoning
A parameterless feature ranking algorithm based on MI

Neurocomputing
COG: local decomposition for rare class analysis

Data Mining and Knowledge Discovery
Effective feature selection scheme using mutual information

Neurocomputing
Supervised feature selection by clustering using conditional mutual information-based distances

Pattern Recognition
Local Causal and Markov Blanket Induction for Causal Discovery and Feature Selection for Classification Part I: Algorithms and Empirical Evaluation

The Journal of Machine Learning Research
Feature subset selection with cumulate conditional mutual information minimization

Expert Systems with Applications: An International Journal
Input feature selection for classification problems

IEEE Transactions on Neural Networks
A comparison of methods for multiclass support vector machines

IEEE Transactions on Neural Networks
Using mutual information for selecting features in supervised neural net learning

IEEE Transactions on Neural Networks

Quality of information-based source assessment and selection

Neurocomputing

Quantified Score

Hi-index	0.01

Visualization

Abstract

Feature selection is one of the core issues in designing pattern recognition systems and has attracted considerable attention in the literature. Most of the feature selection methods in the literature only handle relevance and redundancy analysis from the point of view of the whole class, which neglects the relation of features and the separate classes. In this paper, we propose a novel feature selection framework to explicitly handle the relevance and redundancy analysis for each class label. Then we propose two simple and effective feature selection algorithms based on this framework and Kullback-Leibler divergence. An empirical study is conducted to evaluate the efficiency and effectiveness of our algorithms comparing with five representative feature selection algorithms. Empirical results show that our proposed algorithms are efficient and outperform the selected algorithms in most cases, and show the superiority of our proposed feature selection framework.