Application of global optimization methods to model and feature selection

  • Authors:
  • Abderrahmane Boubezoul;SéBastien Paris

  • Affiliations:
  • Paris-Est University, IFSTTAR, IM, LEPSIS, F-75732 Paris, France;Laboratory of Sciences of Information's and of System, Aix-Marseille University, LSIS/DYNI UMR CNRS 7296, Av Escadrille de Normandie Niemen, 13397 Marseille Cedex 20, France

  • Venue:
  • Pattern Recognition
  • Year:
  • 2012

Quantified Score

Hi-index 0.01

Visualization

Abstract

Many data mining applications involve the task of building a model for predictive classification. The goal of this model is to classify data instances into classes or categories of the same type. The use of variables not related to the classes can reduce the accuracy and reliability of classification or prediction model. Superfluous variables can also increase the costs of building a model particularly on large datasets. The feature selection and hyper-parameters optimization problem can be solved by either an exhaustive search over all parameter values or an optimization procedure that explores only a finite subset of the possible values. The objective of this research is to simultaneously optimize the hyper-parameters and feature subset without degrading the generalization performances of the induction algorithm. We present a global optimization approach based on the use of Cross-Entropy Method to solve this kind of problem.