Learning from the Past with Experiment Databases

Authors:
Joaquin Vanschoren;Bernhard Pfahringer;Geoffrey Holmes
Affiliations:
Computer Science Dept, K.U. Leuven, Leuven, Belgium;Computer Science Dept, University of Waikato, Hamilton, New Zealand;Computer Science Dept, University of Waikato, Hamilton, New Zealand
Venue:
PRICAI '08 Proceedings of the 10th Pacific Rim International Conference on Artificial Intelligence: Trends in Artificial Intelligence
Year:
2008

Citing 8
Cited 1

Very Simple Classification Rules Perform Well on Most Commonly Used Datasets

Machine Learning
The Need for Low Bias Algorithms in Classification Learning from Large Data Sets

PKDD '02 Proceedings of the 6th European Conference on Principles of Data Mining and Knowledge Discovery
Model Class Selection and Construction: Beyond the Procrustean Approach to Machine Learning Applications

Machine Learning and Its Applications, Advanced Lectures
Improved Dataset Characterisation for Meta-learning

DS '02 Proceedings of the 5th International Conference on Discovery Science
Tree induction vs. logistic regression: a learning-curve analysis

The Journal of Machine Learning Research
An empirical comparison of supervised learning algorithms

ICML '06 Proceedings of the 23rd international conference on Machine learning
Data Mining: Practical Machine Learning Tools and Techniques, Second Edition (Morgan Kaufmann Series in Data Management Systems)

Data Mining: Practical Machine Learning Tools and Techniques, Second Edition (Morgan Kaufmann Series in Data Management Systems)
Experiment Databases: Towards an Improved Experimental Methodology in Machine Learning

PKDD 2007 Proceedings of the 11th European conference on Principles and Practice of Knowledge Discovery in Databases

A Community-Based Platform for Machine Learning Experimentation

ECML PKDD '09 Proceedings of the European Conference on Machine Learning and Knowledge Discovery in Databases: Part II

Quantified Score

Hi-index	0.00

Visualization

Abstract

Thousands of Machine Learning research papers contain experimental comparisons that usually have been conducted with a single focus of interest, often losing detailed results after publication. Yet, when collecting all these past experiments in experiment databases, they can readily be reused for additional and possibly much broader investigation. In this paper, we make use of such a database to answer various interesting research questions about learning algorithms and to verify a number of recent studies. Alongside performing elaborate comparisons of algorithms, we also investigate the effects of algorithm parameters and data properties, and seek deeper insights into the behavior of learning algorithms by studying their learning curves and bias-variance profiles.