A study on the use of statistical tests for experimentation with neural networks: Analysis of parametric test conditions and non-parametric tests

Authors:
Julián Luengo;Salvador García;Francisco Herrera
Affiliations:
Department of Computer Science and Artificial Intelligence, University of Granada, C/Daniel Saucedo Aranda S/N, 18071 Granada, Spain;Department of Computer Science and Artificial Intelligence, University of Granada, C/Daniel Saucedo Aranda S/N, 18071 Granada, Spain;Department of Computer Science and Artificial Intelligence, University of Granada, C/Daniel Saucedo Aranda S/N, 18071 Granada, Spain
Venue:
Expert Systems with Applications: An International Journal
Year:
2009

Citing 15
Cited 14

Radial basis functions for multivariable interpolation: a review

Algorithms for approximation
Support-Vector Networks

Machine Learning
Neural networks: a systematic introduction

Neural networks: a systematic introduction
A sequential learning scheme for function approximation using minimal radial basis function neural networks

Neural Computation
Combined 5 × 2 cv F test for comparing supervised classification learning algorithms

Neural Computation
Neural network techniques for financial performance prediction: integrating fundamental and technical analysis

Decision Support Systems - Special issue: Data mining for financial decision making
New Support Vector Algorithms

Neural Computation
Constructing of the risk classification model of cervical cancer by artificial neural network

Expert Systems with Applications: An International Journal
Immune network based ensembles

Neurocomputing
Statistical Comparisons of Classifiers over Multiple Data Sets

The Journal of Machine Learning Research
Comparison of the decision tree, artificial neural network, and linear regression methods based on the number and types of independent variables and sample size

Expert Systems with Applications: An International Journal
Evaluation of ANN and SVM classifiers as predictors to the diagnosis of students with learning disabilities

Expert Systems with Applications: An International Journal
The evaluation of consumer loans using support vector machines

Expert Systems with Applications: An International Journal
On the distribution of performance from multiple neural-network trials

IEEE Transactions on Neural Networks
Statistical analysis of the parameters of a neuro-genetic algorithm

IEEE Transactions on Neural Networks

Analysis of the efficacy of a Two-Stage methodology for ant colony optimization: Case of study with TSP and QAP

Expert Systems with Applications: An International Journal
Original paper: Supervised machine learning and heterotic classification of maize (Zea mays L.) using molecular marker data

Computers and Electronics in Agriculture
Nonparametric statistical analysis of machine learning algorithms for regression problems

KES'10 Proceedings of the 14th international conference on Knowledge-based and intelligent information and engineering systems: Part I
EVOR-STACK: A label-dependent evolutive stacking on remote sensing data fusion

Neurocomputing
Shared domains of competence of approximate learning models using measures of separability of classes

Information Sciences: an International Journal
Application of mixture of experts to construct real estate appraisal models

HAIS'10 Proceedings of the 5th international conference on Hybrid Artificial Intelligence Systems - Volume Part I
Evolutionary algorithms for the design of grid-connected PV-systems

Expert Systems with Applications: An International Journal
An improved swarm optimized functional link artificial neural network (ISO-FLANN) for classification

Journal of Systems and Software
Improving measurement invariance assessments in survey research with missing data by novel artificial neural networks

Expert Systems with Applications: An International Journal
On employing fuzzy modeling algorithms for the valuation of residential premises

Information Sciences: an International Journal
A comparative study of efficient initialization methods for the k-means clustering algorithm

Expert Systems with Applications: An International Journal
Characterization of Concentrating Photovoltaic modules by cooperative competitive Radial Basis Function Networks

Expert Systems with Applications: An International Journal
Developing a hybrid intelligent model for forecasting problems: Case study of tourism demand time series

Knowledge-Based Systems
Investigation of mixture of experts applied to residential premises valuation

ACIIDS'13 Proceedings of the 5th Asian conference on Intelligent Information and Database Systems - Volume Part II

Quantified Score

Hi-index	12.06

Visualization

Abstract

In this paper, we focus on the experimental analysis on the performance in artificial neural networks with the use of statistical tests on the classification task. Particularly, we have studied whether the sample of results from multiple trials obtained by conventional artificial neural networks and support vector machines checks the necessary conditions for being analyzed through parametrical tests. The study is conducted by considering three possibilities on classification experiments: random variation in the selection of test data, the selection of training data and internal randomness in the learning algorithm. The results obtained state that the fulfillment of these conditions are problem-dependent and indefinite, which justifies the need of using non-parametric statistics in the experimental analysis.