Feature subset selection with cumulate conditional mutual information minimization

  • Authors:
  • Yishi Zhang;Zigang Zhang

  • Affiliations:
  • School of Management, Huazhong University of Science and Technology, Wuhan 430074, China;School of Management, Huazhong University of Science and Technology, Wuhan 430074, China

  • Venue:
  • Expert Systems with Applications: An International Journal
  • Year:
  • 2012

Quantified Score

Hi-index 12.05

Visualization

Abstract

Feature selection is one of the core issues in designing pattern recognition and machine learning systems, and has attracted considerable attention in the literature. In this paper, a new feature subset selection algorithm with conditional mutual information is proposed, which firstly guarantees to find a subset of which the mutual information with the class is the same as that of the original set of features, and then eliminates potential redundant features from the view of minimal information loss based on the cumulate conditional mutual information minimization criterion. From the reliability point of view, this criterion can also abate the disturbance caused by sample insufficiency in conditional mutual information estimation. In addition, a fast implementation of conditional mutual information estimation is proposed and used to tackle the computationally intractable problem. Empirical results verify that our algorithm is efficient and achieves better accuracy than several representative feature selection algorithms for three typical classifiers on various datasets.