Taxonomy for characterizing ensemble methods in classification tasks: A review and annotated bibliography

Authors:
Lior Rokach
Affiliations:
Department of Information System Engineering, Ben-Gurion University of the Negev, Israel
Venue:
Computational Statistics & Data Analysis
Year:
2009

Citing 114
Cited 21

The Strength of Weak Learnability

Machine Learning
Multiple binary decision tree classifiers

Pattern Recognition
Evaluation of adaptive mixtures of competing experts

NIPS-3 Proceedings of the 1990 conference on Advances in neural information processing systems 3
Back propagation is sensitive to initial conditions

NIPS-3 Proceedings of the 1990 conference on Advances in neural information processing systems 3
Original Contribution: Stacked generalization

Neural Networks
Decision Combination in Multiple Classifier Systems

IEEE Transactions on Pattern Analysis and Machine Intelligence
Technical Note: Selecting a Classification Method by Cross-Validation

Machine Learning
Hierarchical mixtures of experts and the EM algorithm

Neural Computation
Characterizing the applicability of classification algorithms using meta-level learning

ECML-94 Proceedings of the European conference on machine learning on Machine Learning
Knowledge-based artificial neural networks

Artificial Intelligence
A Method of Combining Multiple Experts for the Recognition of Unconstrained Handwritten Numerals

IEEE Transactions on Pattern Analysis and Machine Intelligence
Recursive Automatic Bias Selection for Classifier Construction

Machine Learning - Special issue on bias evaluation and selection
Bagging predictors

Machine Learning
Error reduction through learning multiple descriptions

Machine Learning
On the Accuracy of Meta-learning for Scalable Data Mining

Journal of Intelligent Information Systems
Combination of Multiple Classifiers Using Local Accuracy Estimates

IEEE Transactions on Pattern Analysis and Machine Intelligence
Graphical models for discovering knowledge

Advances in knowledge discovery and data mining
The Random Subspace Method for Constructing Decision Forests

IEEE Transactions on Pattern Analysis and Machine Intelligence
Generalization performance of support vector machines and other pattern classifiers

Advances in kernel methods
Feature selection for ensembles

AAAI '99/IAAI '99 Proceedings of the sixteenth national conference on Artificial intelligence and the eleventh Innovative applications of artificial intelligence conference innovative applications of artificial intelligence
An Experimental Comparison of Three Methods for Constructing Ensembles of Decision Trees: Bagging, Boosting, and Randomization

Machine Learning
MultiBoosting: A Technique for Combining Boosting and Wagging

Machine Learning
Data mining by attribute decomposition with semiconductor manufacturing case study

Data mining for design and manufacturing
Random Forests

Machine Learning
Ensembling neural networks: many could be better than all

Artificial Intelligence
Forecasting the NYSE composite index with technical analysis, pattern recognizer, neural networks, and genetic algorithm: a case study in romantic decision support

Decision Support Systems
Using Correspondence Analysis to Combine Classifiers

Machine Learning
Pasting Small Votes for Classification in Large Databases and On-Line

Machine Learning
Neural Network Ensembles

IEEE Transactions on Pattern Analysis and Machine Intelligence
Feature Transformation by Function Decomposition

IEEE Intelligent Systems
Stochastic gradient boosting

Computational Statistics & Data Analysis - Nonlinear methods and data mining
Looking for lumps: boosting and bagging for density estimation

Computational Statistics & Data Analysis - Nonlinear methods and data mining
Effect of pruning and early stopping on performance of a boosting ensemble

Computational Statistics & Data Analysis - Nonlinear methods and data mining
Bayesian partition modelling

Computational Statistics & Data Analysis - Nonlinear methods and data mining
Using k-nearest-neighbor classification in the leaves of a tree

Computational Statistics & Data Analysis
Measures of Diversity in Classifier Ensembles and Their Relationship with the Ensemble Accuracy

Machine Learning
Using the Information Structure Model to Compare Profile-Based Information Filtering Systems

Information Retrieval
Rule Induction with CN2: Some Recent Improvements

EWSL '91 Proceedings of the European Working Session on Machine Learning
Variance Optimized Bagging

ECML '02 Proceedings of the 13th European Conference on Machine Learning
How to Make Stacking Better and Faster While Also Taking Care of an Unknown Weakness

ICML '02 Proceedings of the Nineteenth International Conference on Machine Learning
The Alternating Decision Tree Learning Algorithm

ICML '99 Proceedings of the Sixteenth International Conference on Machine Learning
Using Rough Sets Theory and Database Operations to Construct a Good Ensemble of Classifiers for Data Mining Applications

ICDM '01 Proceedings of the 2001 IEEE International Conference on Data Mining
Theory and Applications of Attribute Decomposition

ICDM '01 Proceedings of the 2001 IEEE International Conference on Data Mining
Nearest Neighbors in Random Subspaces

SSPR '98/SPR '98 Proceedings of the Joint IAPR International Workshops on Advances in Pattern Recognition
Ensembles of Learning Machines

WIRN VIETRI 2002 Proceedings of the 13th Italian Workshop on Neural Nets-Revised Papers
Improving Supervised Learning by Feature Decomposition

FoIKS '02 Proceedings of the Second International Symposium on Foundations of Information and Knowledge Systems
Combining Decision Trees and Neural Networks for Drug Discovery

EuroGP '02 Proceedings of the 5th European Conference on Genetic Programming
Combining Multiple K-Nearest Neighbor Classifiers for Text Classification by Reducts

DS '02 Proceedings of the 5th International Conference on Discovery Science
Ensemble Methods in Machine Learning

MCS '00 Proceedings of the First International Workshop on Multiple Classifier Systems
Types of Multinet System

MCS '02 Proceedings of the Third International Workshop on Multiple Classifier Systems
An Empirical Comparison of Pruning Methods for Ensemble Classifiers

IDA '01 Proceedings of the 4th International Conference on Advances in Intelligent Data Analysis
Pruning Adaptive Boosting

ICML '97 Proceedings of the Fourteenth International Conference on Machine Learning
Exploiting unlabeled data in ensemble methods

Proceedings of the eighth ACM SIGKDD international conference on Knowledge discovery and data mining
Ensemble Feature election with the Simple Bayesian Classification in Medical Diagnostics

CBMS '02 Proceedings of the 15th IEEE Symposium on Computer-Based Medical Systems (CBMS'02)
Designing committees of models through deliberate weighting of data points

The Journal of Machine Learning Research
Model selection for medical diagnosis decision support systems

Decision Support Systems
Is Combining Classifiers with Stacking Better than Selecting the Best One?

Machine Learning
Combining diverse neural nets

The Knowledge Engineering Review
Learning Ensembles from Bites: A Scalable and Accurate Approach

The Journal of Machine Learning Research
Multistrategy Ensemble Learning: Reducing Error by Combining Ensemble Learning Techniques

IEEE Transactions on Knowledge and Data Engineering
Ensemble selection from libraries of models

ICML '04 Proceedings of the twenty-first international conference on Machine learning
Detection of land-cover transitions by combining multidate classifiers

Pattern Recognition Letters - Special issue: Pattern recognition for remote sensing (PRRS 2002)
The Dynamics of AdaBoost: Cyclic Behavior and Convergence of Margins

The Journal of Machine Learning Research
Multiknowledge for decision making

Knowledge and Information Systems
Feature selection algorithms for the generation of multiple classifier systems and their application to handwritten word recognition

Pattern Recognition Letters
A decision theoretic approach to combining information filters: An analytical and empirical evaluation

Journal of the American Society for Information Science and Technology
Asymmetric Bagging and Random Subspace for Support Vector Machines-Based Relevance Feedback in Image Retrieval

IEEE Transactions on Pattern Analysis and Machine Intelligence
YALE: rapid prototyping for complex data mining tasks

Proceedings of the 12th ACM SIGKDD international conference on Knowledge discovery and data mining
Decomposition methodology for classification tasks: a meta decomposer framework

Pattern Analysis & Applications
A Comparison of Decision Tree Ensemble Creation Techniques

IEEE Transactions on Pattern Analysis and Machine Intelligence
Classifier evaluation under limited resources

Pattern Recognition Letters
Adaptive fusion and co-operative training for classifier ensembles

Pattern Recognition
Experimental study for the comparison of classifier combination methods

Pattern Recognition
Parallelizing AdaBoost by weights dynamics

Computational Statistics & Data Analysis
Feature set decomposition for decision trees

Intelligent Data Analysis
Negative Samples Analysis in Relevance Feedback

IEEE Transactions on Knowledge and Data Engineering
Decision-tree instance-space decomposition with grouped gain-ratio

Information Sciences: an International Journal
Boosting ridge regression

Computational Statistics & Data Analysis
Classification by ensembles from random partitions of high-dimensional data

Computational Statistics & Data Analysis
Decision trees using model ensemble-based nodes

Pattern Recognition
EROS: Ensemble rough subspaces

Pattern Recognition
Quadratic boosting

Pattern Recognition
Data-driven decomposition for multi-class classification

Pattern Recognition
Empirical characterization of random forest variable importance measures

Computational Statistics & Data Analysis
A local boosting algorithm for solving classification problems

Computational Statistics & Data Analysis
LogitBoost with errors-in-variables

Computational Statistics & Data Analysis
Genetic algorithm-based feature set partitioning for classification problems

Pattern Recognition
Adaptive mixtures of local experts

Neural Computation
Mining manufacturing data using genetic algorithm-based feature set decomposition

International Journal of Intelligent Systems Technologies and Applications
Engineering multiversion neural-net systems

Neural Computation
Detection of unknown computer worms based on behavioral classification of the host

Computational Statistics & Data Analysis
Negation recognition in medical narrative reports

Information Retrieval
NeC4.5: Neural Ensemble Based C4.5

IEEE Transactions on Knowledge and Data Engineering
Improving malware detection by applying multi-inducer ensemble

Computational Statistics & Data Analysis
Collective-agreement-based pruning of ensembles

Computational Statistics & Data Analysis
Java-ML: A Machine Learning Library

The Journal of Machine Learning Research
Solving multiclass learning problems via error-correcting output codes

Journal of Artificial Intelligence Research
Troika - An improved stacking schema for classification tasks

Information Sciences: an International Journal
Constructing diverse classifier ensembles using artificial training examples

IJCAI'03 Proceedings of the 18th international joint conference on Artificial intelligence
Ensemble methods for improving the performance of neighborhood-based collaborative filtering

Proceedings of the third ACM conference on Recommender systems
Inverse boosting for monotone regression functions

Computational Statistics & Data Analysis
Boosting and instability for regression trees

Computational Statistics & Data Analysis
Bundling classifiers by bagging trees

Computational Statistics & Data Analysis
Online adaptive policies for ensemble classifiers

Neurocomputing
Classifier combination based on confidence transformation

Pattern Recognition
Data dependence in combining classifiers

MCS'03 Proceedings of the 4th international conference on Multiple classifier systems
Negative correlation learning and the ambiguity family of ensemble methods

MCS'03 Proceedings of the 4th international conference on Multiple classifier systems
Selective ensemble of decision trees

RSFDGrC'03 Proceedings of the 9th international conference on Rough sets, fuzzy sets, data mining, and granular computing
Bagging, boosting, and C4.S

AAAI'96 Proceedings of the thirteenth national conference on Artificial intelligence - Volume 1
Constructing rough decision forests

RSFDGrC'05 Proceedings of the 10th international conference on Rough Sets, Fuzzy Sets, Data Mining, and Granular Computing - Volume Part II
Multitraining Support Vector Machine for Image Retrieval

IEEE Transactions on Image Processing
Task decomposition and module combination based on class relations: a modular neural network for pattern classification

IEEE Transactions on Neural Networks
A constructive algorithm for training cooperative neural network ensembles

IEEE Transactions on Neural Networks
Efficient classification for multiclass problems using modular neural networks

IEEE Transactions on Neural Networks

Troika - An improved stacking schema for classification tasks

Information Sciences: an International Journal
Ensemble methods for improving the performance of neighborhood-based collaborative filtering

Proceedings of the third ACM conference on Recommender systems
Editorial: A representative bibliography of surveys in the information fusion domain

Information Fusion
Class-specific error bounds for ensemble classifiers

Proceedings of the 16th ACM SIGKDD international conference on Knowledge discovery and data mining
Analysis of bagging ensembles of fuzzy models for premises valuation

ACIIDS'10 Proceedings of the Second international conference on Intelligent information and database systems: Part II
Comparison of bagging, boosting and stacking ensembles applied to real estate appraisal

ACIIDS'10 Proceedings of the Second international conference on Intelligent information and database systems: Part II
Ensemble classification of paired data

Computational Statistics & Data Analysis
Information market based recommender systems fusion

Proceedings of the 2nd International Workshop on Information Heterogeneity and Fusion in Recommender Systems
An experimental study of one- and two-level classifier fusion for different sample sizes

Pattern Recognition Letters
An incremental ensemble of classifiers

Artificial Intelligence Review
Generalised bottom-up pruning: A model level combination of decision trees

Expert Systems with Applications: An International Journal
Assessment of problem modality by differential performance of lexicase selection in genetic programming: a preliminary report

Proceedings of the 14th annual conference companion on Genetic and evolutionary computation
Improving the performance of unit critiquing

UMAP'12 Proceedings of the 20th international conference on User Modeling, Adaptation, and Personalization
Ensemble approaches for regression: A survey

ACM Computing Surveys (CSUR)
A class centric feature and classifier ensemble selection approach for music genre classification

SSPR'12/SPR'12 Proceedings of the 2012 Joint IAPR international conference on Structural, Syntactic, and Statistical Pattern Recognition
The use of artificial-intelligence-based ensembles for intrusion detection: a review

Applied Computational Intelligence and Soft Computing
An intrusion detection and alert correlation approach based on revising probabilistic classifiers using expert knowledge

Applied Intelligence
A survey of multiple classifier systems as hybrid systems

Information Fusion
Ensemble methods for advanced skier days prediction

Expert Systems with Applications: An International Journal
Learning ensemble classifiers via restricted Boltzmann machines

Pattern Recognition Letters
Contrasting meta-learning and hyper-heuristic research: the role of evolutionary algorithms

Genetic Programming and Evolvable Machines

Quantified Score

Hi-index	0.03

Visualization

Abstract

Ensemble methodology, which builds a classification model by integrating multiple classifiers, can be used for improving prediction performance. Researchers from various disciplines such as statistics, pattern recognition, and machine learning have seriously explored the use of ensemble methodology. This paper presents an updated survey of ensemble methods in classification tasks, while introducing a new taxonomy for characterizing them. The new taxonomy, presented from the algorithm designer's point of view, is based on five dimensions: inducer, combiner, diversity, size, and members' dependency. We also propose several selection criteria, presented from the practitioner's point of view, for choosing the most suitable ensemble method.