Tuned data mining: a benchmark study on different tuners

Authors:
Wolfgang Konen;Patrick Koch;Oliver Flasch;Thomas Bartz-Beielstein;Martina Friese;Boris Naujoks
Affiliations:
Cologne University of Applied Sciences, Gummersbach, Germany;Cologne University of Applied Sciences, Gummersbach, Germany;Cologne University of Applied Sciences, Gummersbach, Germany;Cologne University of Applied Sciences, Gummersbach, Germany;Cologne University of Applied Sciences, Gummersbach, Germany;Cologne University of Applied Sciences, Gummersbach, Germany
Venue:
Proceedings of the 13th annual conference on Genetic and evolutionary computation
Year:
2011

Citing 10
Cited 1

A limited memory algorithm for bound constrained optimization

SIAM Journal on Scientific Computing
MetaCost: a general method for making classifiers cost-sensitive

KDD '99 Proceedings of the fifth ACM SIGKDD international conference on Knowledge discovery and data mining
Genetic Algorithms in Search, Optimization and Machine Learning

Genetic Algorithms in Search, Optimization and Machine Learning
Learning with Kernels: Support Vector Machines, Regularization, Optimization, and Beyond

Learning with Kernels: Support Vector Machines, Regularization, Optimization, and Beyond
Random Forests

Machine Learning
A Taxonomy of Hybrid Metaheuristics

Journal of Heuristics
Toward Integrating Feature Selection Algorithms for Classification and Clustering

IEEE Transactions on Knowledge and Data Engineering
Comparing parameter tuning methods for evolutionary algorithms

CEC'09 Proceedings of the Eleventh conference on Congress on Evolutionary Computation
Feature Extraction for Simple Classification

ICPR '10 Proceedings of the 2010 20th International Conference on Pattern Recognition
Experimental Methods for the Analysis of Optimization Algorithms

Experimental Methods for the Analysis of Optimization Algorithms

Efficient sampling and handling of variance in tuning data mining models

PPSN'12 Proceedings of the 12th international conference on Parallel Problem Solving from Nature - Volume Part I

Quantified Score

Hi-index	0.00

Visualization

Abstract

The complex, often redundant and noisy data in real-world data mining (DM) applications frequently lead to inferior results when out-of-the-box DM models are applied. A tuning of parameters is essential to achieve high-quality results. In this work we aim at tuning parameters of the preprocessing and the modeling phase conjointly. The framework TDM (Tuned Data Mining) was developed to facilitate the search for good parameters and the comparison of different tuners. It is shown that tuning is of great importance for high-quality results. Surrogate-model based tuning utilizing the Sequential Parameter Optimization Toolbox (SPOT) is compared with other tuners (CMA-ES, BFGS, LHD) and evidence is found that SPOT is well suited for this task. In benchmark tasks like the Data Mining Cup (DMC) tuned models achieve remarkably better ranks than their untuned counterparts.