Design of Experiments in Computational Intelligence: On the Use of Statistical Inference

Authors:
Salvador García;Francisco Herrera
Affiliations:
Department of Computer Science and Artificial Intelligence, E.T.S.I. Informática, University of Granada, Granada, Spain 18071;Department of Computer Science and Artificial Intelligence, E.T.S.I. Informática, University of Granada, Granada, Spain 18071
Venue:
HAIS '08 Proceedings of the 3rd international workshop on Hybrid Artificial Intelligence Systems
Year:
2008

Citing 13
Cited 0

Neural networks: a systematic introduction

Neural networks: a systematic introduction
Approximate statistical tests for comparing supervised classification learning algorithms

Neural Computation
Genetic Algorithms in Search, Optimization and Machine Learning

Genetic Algorithms in Search, Optimization and Machine Learning
Introduction to Evolutionary Computing

Introduction to Evolutionary Computing
Working Set Selection Using Second Order Information for Training Support Vector Machines

The Journal of Machine Learning Research
Biostatistical Analysis (5th Edition)

Biostatistical Analysis (5th Edition)
Statistical Comparisons of Classifiers over Multiple Data Sets

The Journal of Machine Learning Research
A lot of randomness is hiding in accuracy

Engineering Applications of Artificial Intelligence
Classifier fitness based on accuracy

Evolutionary Computation
Handbook of Parametric and Nonparametric Statistical Procedures

Handbook of Parametric and Nonparametric Statistical Procedures
A study on the use of non-parametric tests for analyzing the evolutionary algorithms' behaviour: a case study on the CEC'2005 Special Session on Real Parameter Optimization

Journal of Heuristics
No free lunch theorems for optimization

IEEE Transactions on Evolutionary Computation
Natural Encoding for Evolutionary Supervised Learning

IEEE Transactions on Evolutionary Computation

Quantified Score

Hi-index	0.00

Visualization

Abstract

The experimental analysis on the performance of a proposed method is a crucial and necessary task to carry out in a research. In this contribution we focus on the use of statistical inference for analyzing the results obtained in a design of experiment within the field of computational intelligence. We present some studies involving a set of techniques in different topics which can be used for doing a rigorous comparison among the algorithms studied in an experimental comparison.Particularly, we study whether the sample of results from multiple trials obtained by the run of several algorithms checks the required conditions for being analyzed through parametric tests. In most of the cases, the results indicate that the fulfillment of these conditions are problem dependent and indefinite, which justifies the need of using nonparametric statistics in the experimental analysis. We show a case study which illustrates the use of nonparametric tests and finally we give some guidelines on the use of nonparametric statistical tests.