Feature selection using linear classifier weights: interaction with classification models

  • Authors:
  • Dunja Mladenić;Janez Brank;Marko Grobelnik;Natasa Milic-Frayling

  • Affiliations:
  • Jožef Stefan Institute, Ljubljana, Slovenia;Jožef Stefan Institute, Ljubljana, Slovenia;Jožef Stefan Institute, Ljubljana, Slovenia;Microsoft Research Ltd, Cambridge, United Kingdom

  • Venue:
  • Proceedings of the 27th annual international ACM SIGIR conference on Research and development in information retrieval
  • Year:
  • 2004

Quantified Score

Hi-index 0.00

Visualization

Abstract

This paper explores feature scoring and selection based on weights from linear classification models. It investigates how these methods combine with various learning models. Our comparative analysis includes three learning algorithms: Naïve Bayes, Perceptron, and Support Vector Machines (SVM) in combination with three feature weighting methods: Odds Ratio, Information Gain, and weights from linear models, the linear SVM and Perceptron. Experiments show that feature selection using weights from linear SVMs yields better classification performance than other feature weighting methods when combined with the three explored learning algorithms. The results support the conjecture that it is the sophistication of the feature weighting method rather than its apparent compatibility with the learning algorithm that improves classification performance.