Improving stability of feature selection methods

  • Authors:
  • Pavel Křížek;Josef Kittler;Václav Hlaváč

  • Affiliations:
  • Czech Technical University in Prague, Center for Machine Perception, Czech Republic;University of Surrey, Centre for Vision, Speech, and Signal Processing, Guildford, United Kingdom;Czech Technical University in Prague, Center for Machine Perception, Czech Republic

  • Venue:
  • CAIP'07 Proceedings of the 12th international conference on Computer analysis of images and patterns
  • Year:
  • 2007

Quantified Score

Hi-index 0.00

Visualization

Abstract

An improper design of feature selection methods can often lead to incorrect conclusions. Moreover, it is not generally realised that functional values of the criterion guiding the search for the best feature set are random variables with some probability distribution. This contribution examines the influence of several estimation techniques on the consistency of the final result. We propose an entropy based measure which can assess the stability of feature selection methods with respect to perturbations in the data. Results show that filters achieve a better stability and performance if more samples are employed for the estimation, i.e., using leave-one-out cross-validation, for instance. However, the best results for wrappers are acquired with the 50/50 holdout validation.