Performance comparisons of neural networks and machine learning techniques: a critical assessment of the methodology

Authors:
Achim Hoffman
Affiliations:
Univ. of New South Wales, Sydney, Australia
Venue:
New learning paradigms in soft computing
Year:
2002

Citing 10
Cited 0

A theory of the learnable

Communications of the ACM
Learnability and the Vapnik-Chervonenkis dimension

Journal of the ACM (JACM)
Empirical methods for artificial intelligence

Empirical methods for artificial intelligence
Bagging predictors

Machine Learning
Wrappers for performance enhancement and oblivious decision graphs

Wrappers for performance enhancement and oblivious decision graphs
Approximate statistical tests for comparing supervised classification learning algorithms

Neural Computation
Machine Learning

Machine Learning
On Comparing Classifiers: Pitfalls toAvoid and a Recommended Approach

Data Mining and Knowledge Discovery
Improved use of continuous attributes in C4.5

Journal of Artificial Intelligence Research
Presenting and analyzing the results of ai experiments: data averaging and data snooping

AAAI'97/IAAI'97 Proceedings of the fourteenth national conference on artificial intelligence and ninth conference on Innovative applications of artificial intelligence

Quantified Score

Hi-index	0.00

Visualization

Abstract

This chapter scrutinises the practice of empirical studies in Neural Network and Machine Learning Research. Recent years saw an increasing sophistication of the statistical evaluation of experiments. This chapter provides a short review of the main ideas of such studies and their statistical evaluation. Further the chapter presents empirical results suggesting that the achievable statistical validity of many studies of the style done in the past is rather limited. This chapter presents a study based on 13 popular datasets from the UCI Machine Learning repository, which demonstrates how careful one has to be when drawing conclusions drawn from such empirical studies.