Feature selection with dynamic mutual information

Authors:
Huawen Liu;Jigui Sun;Lei Liu;Huijie Zhang
Affiliations:
College of Computer Science, Jilin University, Changchun 130012, China;College of Computer Science, Jilin University, Changchun 130012, China and Key Laboratory of Symbolic Computation and Knowledge Engineering of Ministry of Education, Changchun 130012, China;College of Computer Science, Jilin University, Changchun 130012, China and Key Laboratory of Symbolic Computation and Knowledge Engineering of Ministry of Education, Changchun 130012, China;College of Computer, Northeast Normal University, Changchun 130021, China
Venue:
Pattern Recognition
Year:
2009

Citing 49
Cited 42

Instance-Based Learning Algorithms

Machine Learning
Elements of information theory

Elements of information theory
A practical approach to feature selection

ML92 Proceedings of the ninth international workshop on Machine learning
C4.5: programs for machine learning

C4.5: programs for machine learning
Estimating attributes: analysis and extensions of RELIEF

ECML-94 Proceedings of the European conference on machine learning on Machine Learning
Selection of relevant features and examples in machine learning

Artificial Intelligence - Special issue on relevance
Wrappers for feature subset selection

Artificial Intelligence - Special issue on relevance
On the approximability of minimizing nonzero variables or unsatisfied relations in linear systems

Theoretical Computer Science
Data mining: practical machine learning tools and techniques with Java implementations

Data mining: practical machine learning tools and techniques with Java implementations
Adaptive Intrusion Detection: A Data Mining Approach

Artificial Intelligence Review - Issues on the application of data mining
Unsupervised Feature Selection Using Feature Similarity

IEEE Transactions on Pattern Analysis and Machine Intelligence
Neural Networks for Pattern Recognition

Neural Networks for Pattern Recognition
Rough Sets: Theoretical Aspects of Reasoning about Data

Rough Sets: Theoretical Aspects of Reasoning about Data
A Formalism for Relevance and Its Application in Feature Subset Selection

Machine Learning
Input Feature Selection by Mutual Information Based on Parzen Window

IEEE Transactions on Pattern Analysis and Machine Intelligence
Unsupervised Feature Selection Applied to Content-Based Retrieval of Lung Images

IEEE Transactions on Pattern Analysis and Machine Intelligence
Feature selection for high-dimensional genomic microarray data

ICML '01 Proceedings of the Eighteenth International Conference on Machine Learning
Efficient content-based image retrieval using automatic feature selection

ISCV '95 Proceedings of the International Symposium on Computer Vision
Theoretical and Empirical Analysis of ReliefF and RReliefF

Machine Learning
An introduction to variable and feature selection

The Journal of Machine Learning Research
An extensive empirical study of feature selection metrics for text classification

The Journal of Machine Learning Research
Benchmarking Attribute Selection Techniques for Discrete Class Data Mining

IEEE Transactions on Knowledge and Data Engineering
Selective Sampling for Nearest Neighbor Classifiers

Machine Learning
Consistency-based search in feature selection

Artificial Intelligence
Fast Branch & Bound Algorithms for Optimal Feature Selection

IEEE Transactions on Pattern Analysis and Machine Intelligence
Feature Selection for Unsupervised Learning

The Journal of Machine Learning Research
Feature selection with conditional mutual information maximin in text categorization

Proceedings of the thirteenth ACM international conference on Information and knowledge management
A selective sampling approach to active feature selection

Artificial Intelligence
Efficient Feature Selection via Analysis of Relevance and Redundancy

The Journal of Machine Learning Research
Fast Binary Feature Selection with Conditional Mutual Information

The Journal of Machine Learning Research
Toward Integrating Feature Selection Algorithms for Classification and Clustering

IEEE Transactions on Knowledge and Data Engineering
Feature Selection Based on Mutual Information: Criteria of Max-Dependency, Max-Relevance, and Min-Redundancy

IEEE Transactions on Pattern Analysis and Machine Intelligence
A New Dependency and Correlation Analysis for Features

IEEE Transactions on Knowledge and Data Engineering
Combined SVM-Based Feature Selection and Classification

Machine Learning
A fast and effective method to find correlations among attributes in databases

Data Mining and Knowledge Discovery
A Stochastic Algorithm for Feature Selection in Pattern Recognition

The Journal of Machine Learning Research
Spectral feature selection for supervised and unsupervised learning

Proceedings of the 24th international conference on Machine learning
A hybrid genetic algorithm for feature selection wrapper based on mutual information

Pattern Recognition Letters
Feature selection methods for text classification

Proceedings of the 13th ACM SIGKDD international conference on Knowledge discovery and data mining
Active learning for logistic regression: an evaluation

Machine Learning
Feature selection using correlation fractal dimension: Issues and applications in binary classification problems

Applied Soft Computing
A new mutual information based measure for feature selection

Intelligent Data Analysis
Invariant optimal feature selection: A distance discriminant and feature ranking based solution

Pattern Recognition
Constraint Score: A new filter method for feature selection with pairwise constraints

Pattern Recognition
A review of feature selection techniques in bioinformatics

Bioinformatics
A unified framework for semi-supervised dimensionality reduction

Pattern Recognition
Effective feature selection scheme using mutual information

Neurocomputing
Estimating continuous distributions in Bayesian classifiers

UAI'95 Proceedings of the Eleventh conference on Uncertainty in artificial intelligence
Using mutual information for selecting features in supervised neural net learning

IEEE Transactions on Neural Networks

Boosting feature selection using information metric for classification

Neurocomputing
Ensemble gene selection by grouping for microarray data classification

Journal of Biomedical Informatics
Bagging Constraint Score for feature selection with pairwise constraints

Pattern Recognition
Performance evaluation of multiple classification of the ultrasonic supraspinatus images by using ML, RBFNN and SVM classifiers

Expert Systems with Applications: An International Journal
Ensemble gene selection for cancer classification

Pattern Recognition
Feature selection for Bayesian network classifiers using the MDL-FS score

International Journal of Approximate Reasoning
Textural feature selection by joint mutual information based on Gaussian mixture model for multispectral image classification

Pattern Recognition Letters
A novel information theoretic-interact algorithm (IT-IN) for feature selection using three machine learning algorithms

Expert Systems with Applications: An International Journal
Diagnose the mild cognitive impairment by constructing Bayesian network with missing data

Expert Systems with Applications: An International Journal
Selection of relevant variables for industrial process modeling by combining experimental data sensitivity and human knowledge

Engineering Applications of Artificial Intelligence
BASSUM: A Bayesian semi-supervised method for classification feature selection

Pattern Recognition
Using support vector machines with a novel hybrid feature selection method for diagnosis of erythemato-squamous diseases

Expert Systems with Applications: An International Journal
Estimating redundancy information of selected features in multi-dimensional pattern classification

Pattern Recognition Letters
Sparse kernel density estimations and its application in variable selection based on quadratic Renyi entropy

Neurocomputing
Correntropy based feature selection using binary projection

Pattern Recognition
mr2PSO: A maximum relevance minimum redundancy feature selection method based on swarm intelligence for support vector machine classification

Information Sciences: an International Journal
Combining linear dimensionality reduction and locality preserving projections with feature selection for recognition tasks

ACIVS'11 Proceedings of the 13th international conference on Advanced concepts for intelligent vision systems
Feature subset selection wrapper based on mutual information and rough sets

Expert Systems with Applications: An International Journal
An effective feature selection method using dynamic information criterion

AICI'11 Proceedings of the Third international conference on Artificial intelligence and computational intelligence - Volume Part I
ECG arrhythmia recognition via a neuro-SVM-KNN hybrid classifier with virtual QRS image-based geometrical features

Expert Systems with Applications: An International Journal
Feature selection using hierarchical feature clustering

Proceedings of the 20th ACM international conference on Information and knowledge management
Probability density estimation based on nonparametric local kernel regression

ISNN'10 Proceedings of the 7th international conference on Advances in Neural Networks - Volume Part I
Noisy data elimination using mutual k-nearest neighbor for classification mining

Journal of Systems and Software
Feature evaluation and selection with cooperative game theory

Pattern Recognition
Novel hybrid feature selection algorithms for diagnosing erythemato-squamous diseases

HIS'12 Proceedings of the First international conference on Health Information Science
An information theoretic approach for feature selection

Security and Communication Networks
A reduct derived from feature selection

Pattern Recognition Letters
Optimizing the operating conditions in a high precision industrial process using soft computing techniques

Expert Systems: The Journal of Knowledge Engineering
Modeling spectral data based on mutual information and kernel extreme learning machines

ISNN'12 Proceedings of the 9th international conference on Advances in Neural Networks - Volume Part I
DBFS: An effective Density Based Feature Selection scheme for small sample size and high dimensional imbalanced data sets

Data & Knowledge Engineering
Neighborhood effective information ratio for hybrid feature subset evaluation and selection

Neurocomputing
A novel probabilistic feature selection method for text classification

Knowledge-Based Systems
Feature selection using dynamic weights for classification

Knowledge-Based Systems
Information-theoretic selection of high-dimensional spectral features for structural recognition

Computer Vision and Image Understanding
Novel soft subspace clustering with multi-objective evolutionary approach for high-dimensional data

Pattern Recognition
Resampling methods for quality assessment of classifier performance and optimal number of features

Signal Processing
Feature Interaction Maximisation

Pattern Recognition Letters
Spatial distance join based feature selection

Engineering Applications of Artificial Intelligence
The impact of preprocessing on text classification

Information Processing and Management: an International Journal
Robust feature selection based on regularized brownboost loss

Knowledge-Based Systems
A novel feature selection method and its application

Journal of Intelligent Information Systems
Time-efficient estimation of conditional mutual information for variable selection in classification

Computational Statistics & Data Analysis

Quantified Score

Hi-index	0.01

Visualization

Abstract

Feature selection plays an important role in data mining and pattern recognition, especially for large scale data. During past years, various metrics have been proposed to measure the relevance between different features. Since mutual information is nonlinear and can effectively represent the dependencies of features, it is one of widely used measurements in feature selection. Just owing to these, many promising feature selection algorithms based on mutual information with different parameters have been developed. In this paper, at first a general criterion function about mutual information in feature selector is introduced, which can bring most information measurements in previous algorithms together. In traditional selectors, mutual information is estimated on the whole sampling space. This, however, cannot exactly represent the relevance among features. To cope with this problem, the second purpose of this paper is to propose a new feature selection algorithm based on dynamic mutual information, which is only estimated on unlabeled instances. To verify the effectiveness of our method, several experiments are carried out on sixteen UCI datasets using four typical classifiers. The experimental results indicate that our algorithm achieved better results than other methods in most cases.