Advanced nonparametric tests for multiple comparisons in the design of experiments in computational intelligence and data mining: Experimental analysis of power

Authors:
Salvador García;Alberto Fernández;Julián Luengo;Francisco Herrera
Affiliations:
Department of Computer Science, University of Jaén, Spain;Department of Computer Science and Artificial Intelligence, University of Granada, Spain;Department of Computer Science and Artificial Intelligence, University of Granada, Spain;Department of Computer Science and Artificial Intelligence, University of Granada, Spain
Venue:
Information Sciences: an International Journal
Year:
2010

Citing 27
Cited 90

Approximate statistical tests for comparing supervised classification learning algorithms

Neural Computation
A hybrid decision tree/genetic algorithm method for data mining

Information Sciences: an International Journal - Special issue: Soft computing data mining
Evolving RBF neural networks for time-series forecasting with EvRBF

Information Sciences—Informatics and Computer Science: An International Journal - Special issue: Informatics and computer science intelligent systems applications
Data Mining: Concepts and Techniques

Data Mining: Concepts and Techniques
Introduction to Data Mining, (First Edition)

Introduction to Data Mining, (First Edition)
Handbook of Mathematical Functions, With Formulas, Graphs, and Mathematical Tables,

Handbook of Mathematical Functions, With Formulas, Graphs, and Mathematical Tables,
Statistical Comparisons of Classifiers over Multiple Data Sets

The Journal of Machine Learning Research
A discretization algorithm based on Class-Attribute Contingency Coefficient

Information Sciences: an International Journal
Designing of classifiers based on immune principles and fuzzy rules

Information Sciences: an International Journal
A general framework for statistical performance comparison of evolutionary computation algorithms

Information Sciences: an International Journal
Greedy regression ensemble selection: Theory and an application to water quality prediction

Information Sciences: an International Journal
KEEL: a software tool to assess evolutionary algorithms for data mining problems

Soft Computing - A Fusion of Foundations, Methodologies and Applications - Special Issue on Evolutionary and Metaheuristics based Data Mining (EMBDM); Guest Editors: José A. Gámez, María J. del Jesús, José M. Puerta
Evolutionary product-unit neural networks classifiers

Neurocomputing
Discretization for naive-Bayes learning: managing discretization bias and variance

Machine Learning
Incremental construction of classifier and discriminant ensembles

Information Sciences: an International Journal
Contingency matrix theory: Statistical dependence in a contingency table

Information Sciences: an International Journal
Handbook of Parametric and Nonparametric Statistical Procedures

Handbook of Parametric and Nonparametric Statistical Procedures
A study of statistical techniques and performance measures for genetics-based machine learning: accuracy and interpretability

Soft Computing - A Fusion of Foundations, Methodologies and Applications
Computational Intelligence: An Introduction

Computational Intelligence: An Introduction
A study on the use of non-parametric tests for analyzing the evolutionary algorithms' behaviour: a case study on the CEC'2005 Special Session on Real Parameter Optimization

Journal of Heuristics
Bloat control and generalization pressure using the minimum description length principle for a pittsburgh approach learning classifier system

IWLCS'03-05 Proceedings of the 2003-2005 international conference on Learning classifier systems
Improving the performance of a pittsburgh learning classifier system using a default rule

IWLCS'03-05 Proceedings of the 2003-2005 international conference on Learning classifier systems
No free lunch theorems for optimization

IEEE Transactions on Evolutionary Computation
Data mining with an ant colony optimization algorithm

IEEE Transactions on Evolutionary Computation
Using evolutionary algorithms as instance selection for data reduction in KDD: an experimental study

IEEE Transactions on Evolutionary Computation
Hybridization of fuzzy GBML approaches for pattern classification problems

IEEE Transactions on Systems, Man, and Cybernetics, Part B: Cybernetics
Support vector learning for fuzzy rule-based classification systems

IEEE Transactions on Fuzzy Systems

Improving the performance of fuzzy rule-based classification systems with interval-valued fuzzy sets and genetic amplitude tuning

Information Sciences: an International Journal
Analysis of an evolutionary RBFN design algorithm, CO2RBFN, for imbalanced data sets

Pattern Recognition Letters
Solving multi-class problems with linguistic fuzzy rule based classification systems based on pairwise learning and preference relations

Fuzzy Sets and Systems
A heuristic-based hybrid genetic-variable neighborhood search algorithm for task scheduling in heterogeneous multiprocessor system

Information Sciences: an International Journal
Differential evolution for optimizing the positioning of prototypes in nearest neighbor classification

Pattern Recognition
IPADE: iterative prototype adjustment for nearest neighbor classification

IEEE Transactions on Neural Networks
Evolutionary selection of hyperrectangles in nested generalized exemplar learning

Applied Soft Computing
An overview of ensemble methods for binary classifiers in multi-class problems: Experimental study on one-vs-one and one-vs-all schemes

Pattern Recognition
Particle swarm optimisation with simple and efficient neighbourhood search strategies

International Journal of Innovative Computing and Applications
Intelligent data analysis and model interpretation with spectral analysis fuzzy symbolic modeling

International Journal of Approximate Reasoning
A genetic tuning to improve the performance of Fuzzy Rule-Based Classification Systems with Interval-Valued Fuzzy Sets: Degree of ignorance and lateral position

International Journal of Approximate Reasoning
Empirical comparison of bagging ensembles created using weak learners for a regression problem

ACIIDS'11 Proceedings of the Third international conference on Intelligent information and database systems - Volume Part II
A study on the medium-term forecasting using exogenous variable selection of the extra-virgin olive oil with soft computing methods

Applied Intelligence
Enhancing particle swarm optimization using generalized opposition-based learning

Information Sciences: an International Journal
Feature selection in regression tasks using conditional mutual information

IbPRIA'11 Proceedings of the 5th Iberian conference on Pattern recognition and image analysis
Enhancing IPADE algorithm with a different individual codification

HAIS'11 Proceedings of the 6th international conference on Hybrid artificial intelligent systems - Volume Part II
Evolutionary-based selection of generalized instances for imbalanced classification

Knowledge-Based Systems
Mining interestingness measures for string pattern mining

Knowledge-Based Systems
EVOR-STACK: A label-dependent evolutive stacking on remote sensing data fusion

Neurocomputing
Enhancing evolutionary instance selection algorithms by means of fuzzy rough set based feature selection

Information Sciences: an International Journal
C4.5 consolidation process: an alternative to intelligent oversampling methods in class imbalance problems

CAEPIA'11 Proceedings of the 14th international conference on Advances in artificial intelligence: spanish association for artificial intelligence
Cost-conscious comparison of supervised learning algorithms over multiple data sets

Pattern Recognition
Eigenclassifiers for combining correlated classifiers

Information Sciences: an International Journal
Analysis of preprocessing vs. cost-sensitive learning for imbalanced classification. Open problems on intrinsic data characteristics

Expert Systems with Applications: An International Journal
On the use of multi neighbourhood structures within a Tabu-based memetic approach to university timetabling problems

Information Sciences: an International Journal
On the use of cross-validation for time series predictor evaluation

Information Sciences: an International Journal
Evolutionary algorithms for the design of grid-connected PV-systems

Expert Systems with Applications: An International Journal
New results on minimum error entropy decision trees

CIARP'11 Proceedings of the 16th Iberoamerican Congress conference on Progress in Pattern Recognition, Image Analysis, Computer Vision, and Applications
Exploring the behaviour of base classifiers in credit scoring ensembles

Expert Systems with Applications: An International Journal
An improved swarm optimized functional link artificial neural network (ISO-FLANN) for classification

Journal of Systems and Software
Effective search for Pittsburgh learning classifier systems via estimation of distribution algorithms

Information Sciences: an International Journal
A real time traffic simulator utilizing an adaptive fuzzy inference mechanism by tuning fuzzy parameters

Applied Intelligence
Biological invasion-inspired migration in distributed evolutionary algorithms

Information Sciences: an International Journal
Generalised opposition-based differential evolution: an experimental study

International Journal of Computer Applications in Technology
Unsupervised classification of audio signals by self-organizing maps and bayesian labeling

HAIS'12 Proceedings of the 7th international conference on Hybrid Artificial Intelligent Systems - Volume Part I
Hybrid artificial intelligence approaches on vehicle routing problem in logistics distribution

HAIS'12 Proceedings of the 7th international conference on Hybrid Artificial Intelligent Systems - Volume Part I
Real-World problem for checking the sensitiveness of evolutionary algorithms to the choice of the random number generator

HAIS'12 Proceedings of the 7th international conference on Hybrid Artificial Intelligent Systems - Volume Part I
Optimization of neuro-coefficient smooth transition autoregressive models using differential evolution

HAIS'12 Proceedings of the 7th international conference on Hybrid Artificial Intelligent Systems - Volume Part I
PATMAP: polyadenylation site identification from next-generation sequencing data

HAIS'12 Proceedings of the 7th international conference on Hybrid Artificial Intelligent Systems - Volume Part I
On how percolation threshold affects PSO performance

HAIS'12 Proceedings of the 7th international conference on Hybrid Artificial Intelligent Systems - Volume Part I
Supervision strategy of a solar volumetric receiver using NN and rule based techniques

HAIS'12 Proceedings of the 7th international conference on Hybrid Artificial Intelligent Systems - Volume Part I
A predictive evolutionary algorithm for dynamic constrained inverse kinematics problems

HAIS'12 Proceedings of the 7th international conference on Hybrid Artificial Intelligent Systems - Volume Part I
Reasoning with qualitative velocity: towards a hybrid approach

HAIS'12 Proceedings of the 7th international conference on Hybrid Artificial Intelligent Systems - Volume Part I
Research of neural network classifier based on FCM and PSO for breast cancer classification

HAIS'12 Proceedings of the 7th international conference on Hybrid Artificial Intelligent Systems - Volume Part I
A co-evolutionary framework for nearest neighbor enhancement: combining instance and feature weighting with instance selection

HAIS'12 Proceedings of the 7th international conference on Hybrid Artificial Intelligent Systems - Volume Part II
On employing fuzzy modeling algorithms for the valuation of residential premises

Information Sciences: an International Journal
Evolving cognitive and social experience in Particle Swarm Optimization through Differential Evolution: A hybrid approach

Information Sciences: an International Journal
Integrating a differential evolution feature weighting scheme into prototype generation

Neurocomputing
On the use of data filtering techniques for credit risk prediction with instance-based models

Expert Systems with Applications: An International Journal
Generation of a probabilistic fuzzy rule base by learning from examples

Information Sciences: an International Journal
Dynamic fusion method using Localized Generalization Error Model

Information Sciences: an International Journal
The heterogeneous multi-factory production network scheduling with adaptive communication policy and parallel machine

Information Sciences: an International Journal
Constraint projections for semi-supervised affinity propagation

Knowledge-Based Systems
A novel hybrid genetic algorithm with Tabu search for optimizing multi-dimensional functions and point pattern recognition

Information Sciences: an International Journal
Parallel differential evolution with self-adapting control parameters and generalized opposition-based learning for solving high-dimensional optimization problems

Journal of Parallel and Distributed Computing
Comparison of metaheuristic strategies for peakbin selection in proteomic mass spectrometry data

Information Sciences: an International Journal
The CASH algorithm-cost-sensitive attribute selection using histograms

Information Sciences: an International Journal
HyDR-MI: A hybrid algorithm to reduce dimensionality in multiple instance learning

Information Sciences: an International Journal
Repairing fractures between data using genetic programming-based feature extraction: A case study in cancer diagnosis

Information Sciences: an International Journal
Diversity enhanced particle swarm optimization with neighborhood search

Information Sciences: an International Journal
Differential Evolution for automatic rule extraction from medical databases

Applied Soft Computing
A modification to classical evolutionary programming by shifting strategy parameters

Applied Intelligence
Developing Domain-Knowledge Evolutionary Algorithms for Network-on-Chip Application Mapping

Microprocessors & Microsystems
Protein structure prediction using distributed parallel particle swarm optimization

Natural Computing: an international journal
A harmony search algorithm for nurse rostering problems

Information Sciences: an International Journal
Discovering subgroups by means of genetic programming

EuroGP'13 Proceedings of the 16th European conference on Genetic Programming
An interpretable classification rule mining algorithm

Information Sciences: an International Journal
Effective search for genetic-based machine learning systems via estimation of distribution algorithms and embedded feature reduction techniques

Neurocomputing
Adaptive Memetic Differential Evolution with Global and Local neighborhood-based mutation operators

Information Sciences: an International Journal
FRPS: A Fuzzy Rough Prototype Selection method

Pattern Recognition
Tackling the problem of classification with noisy data using Multiple Classifier Systems: Analysis of the performance and robustness

Information Sciences: an International Journal
EUSBoost: Enhancing ensembles for highly imbalanced data-sets by evolutionary undersampling

Pattern Recognition
Classifier fusion with interval-valued weights

Pattern Recognition Letters
Bootstrap analysis of multiple repetitions of experiments using an interval-valued multiple comparison procedure

Journal of Computer and System Sciences
A medical diagnosis support system based on automatic knowledge extraction from databases through differential evolution

International Journal of Data Mining and Bioinformatics
Evaluating the perceived and estimated quality in use of Web 2.0 applications

Journal of Systems and Software
MOGA-based fuzzy data mining with taxonomy

Knowledge-Based Systems
An efficient adaptive fuzzy inference system for complex and high dimensional regression problems in linguistic fuzzy modelling

Knowledge-Based Systems
MEFES: An evolutionary proposal for the detection of exceptions in subgroup discovery. An application to Concentrating Photovoltaic Technology

Knowledge-Based Systems
Multiobjective genetic classifier selection for random oracles fuzzy rule-based classifier ensembles: How beneficial is the additional diversity?

Knowledge-Based Systems
A two-phase heuristic for the energy-efficient scheduling of independent tasks on computational grids

Cluster Computing
On the monitoring task of solar thermal fluid transfer systems using NN based models and rule based techniques

Engineering Applications of Artificial Intelligence
Residual-based fault detection using soft computing techniques for condition monitoring at rolling mills

Information Sciences: an International Journal
Fuzzy nearest neighbor algorithms: Taxonomy, experimental analysis and prospects

Information Sciences: an International Journal
CALA: An unsupervised URL-based web page classification system

Knowledge-Based Systems
Comparing evolutionary algorithms and particle filters for Markerless Human Motion Capture

Applied Soft Computing
Population based Local Search for university course timetabling problems

Applied Intelligence
Regularized Gaussian Mixture Model based discretization for gene expression data association mining

Applied Intelligence
On the characterization of noise filters for self-training semi-supervised in nearest neighbor classification

Neurocomputing
Distributed Estimation of Distribution Algorithms for continuous optimization: How does the exchanged information influence their behavior?

Information Sciences: an International Journal

Quantified Score

Hi-index	0.08

Visualization

Abstract

Experimental analysis of the performance of a proposed method is a crucial and necessary task in an investigation. In this paper, we focus on the use of nonparametric statistical inference for analyzing the results obtained in an experiment design in the field of computational intelligence. We present a case study which involves a set of techniques in classification tasks and we study a set of nonparametric procedures useful to analyze the behavior of a method with respect to a set of algorithms, such as the framework in which a new proposal is developed. Particularly, we discuss some basic and advanced nonparametric approaches which improve the results offered by the Friedman test in some circumstances. A set of post hoc procedures for multiple comparisons is presented together with the computation of adjusted p-values. We also perform an experimental analysis for comparing their power, with the objective of detecting the advantages and disadvantages of the statistical tests described. We found that some aspects such as the number of algorithms, number of data sets and differences in performance offered by the control method are very influential in the statistical tests studied. Our final goal is to offer a complete guideline for the use of nonparametric statistical procedures for performing multiple comparisons in experimental studies.