A parameterless feature ranking algorithm based on MI

  • Authors:
  • Jin-Jie Huang;Yun-Ze Cai;Xiao-Ming Xu

  • Affiliations:
  • Department of Automation, Harbin University of Science and Technology, Xuefu Road 52, Harbin, 150080, China and Department of Automation, Shanghai Jiao Tong University, Dongchuan Road 800, Shangha ...;Department of Automation, Shanghai Jiao Tong University, Dongchuan Road 800, Shanghai 200240, China;Department of Automation, Shanghai Jiao Tong University, Dongchuan Road 800, Shanghai 200240, China and Shanghai Academy of Systems Science, University of Shanghai for Science and Technology, Jung ...

  • Venue:
  • Neurocomputing
  • Year:
  • 2008

Quantified Score

Hi-index 0.01

Visualization

Abstract

A parameterless feature ranking approach is presented for feature selection in the pattern classification task. Compared with Battiti's mutual information feature selection (MIFS) and Kwak and Choi's MIFS-U methods, the proposed method derives an estimation of the conditional MI between the candidate feature f"i and the output class C given the subset of selected features S, i.e. I(C;f"i|S), without any parameters like @b in MIFS and MIFS-U methods to be preset. Thus, the intractable problem can be avoided completely, which is how to choose an appropriate value for @b to achieve the tradeoff between the relevance to the output classes and the redundancy with the already-selected features. Furthermore, a modified greedy feature selection algorithm called the second order MI feature selection approach (SOMIFS) is proposed. Experimental results demonstrate the superiority of SOMIFS in terms of both synthetic and benchmark data sets.