Incremental construction of classifier and discriminant ensembles

Authors:
Aydın Ulaş;Murat Semerci;Olcay Taner Yıldız;Ethem Alpaydın
Affiliations:
Department of Computer Engineering, Boğaziçi University, TR-34342 Bebek, Istanbul, Turkey;Department of Computer Science, Rensselaer Polytechnic Institute, 110 8th Street, Lally Hall, Troy, NY 12180-3590, USA;Department of Computer Engineering, Işık University, TR-34980 Şile, Istanbul, Turkey;Department of Computer Engineering, Boğaziçi University, TR-34342 Bebek, Istanbul, Turkey
Venue:
Information Sciences: an International Journal
Year:
2009

Citing 45
Cited 27

Original Contribution: Stacked generalization

Neural Networks
Bagging predictors

Machine Learning
On Combining Classifiers

IEEE Transactions on Pattern Analysis and Machine Intelligence
The Random Subspace Method for Constructing Decision Forests

IEEE Transactions on Pattern Analysis and Machine Intelligence
Approximate statistical tests for comparing supervised classification learning algorithms

Neural Computation
Optimal Linear Combination of Neural Networks for Improving Classification Performance

IEEE Transactions on Pattern Analysis and Machine Intelligence
Combining predictors: comparison of five meta machine learning methods

Information Sciences: an International Journal
Experimental evaluation of expert fusion strategies

Pattern Recognition Letters - Special issue on pattern recognition in practice VI
A Theoretical Study on Six Classifier Fusion Strategies

IEEE Transactions on Pattern Analysis and Machine Intelligence
Cost complexity-based pruning of ensemble classifiers

Knowledge and Information Systems
Ensembling neural networks: many could be better than all

Artificial Intelligence
Using Correspondence Analysis to Combine Classifiers

Machine Learning
Measures of Diversity in Classifier Ensembles and Their Relationship with the Ensemble Accuracy

Machine Learning
Linear Discriminant Trees

ICML '00 Proceedings of the Seventeenth International Conference on Machine Learning
Combining Multiple Representations and Classifiers for Pen-based Handwritten Digit Recognitio

ICDAR '97 Proceedings of the 4th International Conference on Document Analysis and Recognition
Pruning Adaptive Boosting

ICML '97 Proceedings of the Fourteenth International Conference on Machine Learning
Clustering ensembles of neural network models

Neural Networks
Combining Pattern Classifiers: Methods and Algorithms

Combining Pattern Classifiers: Methods and Algorithms
Genetic programming in classifying large-scale data: an ensemble method

Information Sciences: an International Journal - Special issue: Soft computing data mining
Ensemble selection from libraries of models

ICML '04 Proceedings of the twenty-first international conference on Machine learning
Combined 5 × 2 cv F Test for Comparing Supervised Classification Learning Algorithms

Neural Computation
Cost-conscious classifier ensembles

Pattern Recognition Letters
A New Objective Function for Ensemble Selection in Random Subspaces

ICPR '06 Proceedings of the 18th International Conference on Pattern Recognition - Volume 04
On the impact of fusion strategies on classification errors for large ensembles of classifiers

Pattern Recognition
Using boosting to prune bagging ensembles

Pattern Recognition Letters
Experimental study for the comparison of classifier combination methods

Pattern Recognition
Trainable fusion rules. I. Large sample size case

Neural Networks
Trainable fusion rules. II. Small sample-size effects

Neural Networks
Statistical Comparisons of Classifiers over Multiple Data Sets

The Journal of Machine Learning Research
Ensemble Pruning Via Semi-definite Programming

The Journal of Machine Learning Research
Classifier Ensembles with a Random Linear Oracle

IEEE Transactions on Knowledge and Data Engineering
To Select or To Weigh: A Comparative Study of Linear Combination Schemes for SuperParent-One-Dependence Estimators

IEEE Transactions on Knowledge and Data Engineering
Adaptive mixtures of local experts

Neural Computation
Engineering multiversion neural-net systems

Neural Computation
RotBoost: A technique for combining Rotation Forest and AdaBoost

Pattern Recognition Letters
A Theoretical Analysis of Bagging as a Linear Combination of Classifiers

IEEE Transactions on Pattern Analysis and Machine Intelligence
A dynamic overproduce-and-choose strategy for the selection of classifier ensembles

Pattern Recognition
An Analysis of Ensemble Pruning Techniques Based on Ordered Aggregation

IEEE Transactions on Pattern Analysis and Machine Intelligence
Issues in stacked generalization

Journal of Artificial Intelligence Research
Modeling the concept of majority opinion in group decision making

Information Sciences: an International Journal
Letters: Bio-molecular cancer prediction with random subspace ensembles of support vector machines

Neurocomputing
Classifier combination based on confidence transformation

Pattern Recognition
Score normalization in multimodal biometric systems

Pattern Recognition
Representation Plurality and Fusion for 3-D Face Recognition

IEEE Transactions on Systems, Man, and Cybernetics, Part B: Cybernetics
A constructive algorithm for training cooperative neural network ensembles

IEEE Transactions on Neural Networks

Troika - An improved stacking schema for classification tasks

Information Sciences: an International Journal
A forecasting solution to the oil spill problem based on a hybrid intelligent system

Information Sciences: an International Journal
Advanced nonparametric tests for multiple comparisons in the design of experiments in computational intelligence and data mining: Experimental analysis of power

Information Sciences: an International Journal
A combinational incremental ensemble of classifiers as a technique for predicting students' performance in distance education

Knowledge-Based Systems
So near and yet so far: New insight into properties of some well-known classifier paradigms

Information Sciences: an International Journal
A dynamic classifier ensemble selection approach for noise data

Information Sciences: an International Journal
Classifier fusion in the Dempster--Shafer framework using optimized t-norm based combination rules

International Journal of Approximate Reasoning
Adaptive ROC-based ensembles of HMMs applied to anomaly detection

Pattern Recognition
A multiple Kernel learning algorithm for cell nucleus classification of renal cell carcinoma

ICIAP'11 Proceedings of the 16th international conference on Image analysis and processing: Part I
An incremental ensemble of classifiers

Artificial Intelligence Review
A diversity-driven structure learning algorithm for building hierarchical neuro-fuzzy classifiers

Information Sciences: an International Journal
Dynamic selection of generative-discriminative ensembles for off-line signature verification

Pattern Recognition
Eigenclassifiers for combining correlated classifiers

Information Sciences: an International Journal
A Two Stage Sequential Ensemble Applied to the Classification of Alzheimer's Disease Based on MRI Features

Neural Processing Letters
User preferences based software defect detection algorithms selection using MCDM

Information Sciences: an International Journal
Supervised subspace projections for constructing ensembles of classifiers

Information Sciences: an International Journal
Evolution of heterogeneous ensembles through dynamic particle swarm optimization for video-based face recognition

Pattern Recognition
Training regression ensembles by sequential target correction and resampling

Information Sciences: an International Journal
A retrieval system from inverse synthetic aperture radar images: Application to radar target recognition

Information Sciences: an International Journal
Using diversity in classifier set selection for arabic handwritten recognition

MCS'10 Proceedings of the 9th international conference on Multiple Classifier Systems
A multi-classifier system for off-line signature verification based on dissimilarity representation

MCS'10 Proceedings of the 9th international conference on Multiple Classifier Systems
Multimodal schizophrenia detection by multiclassification analysis

CIARP'11 Proceedings of the 16th Iberoamerican Congress conference on Progress in Pattern Recognition, Image Analysis, Computer Vision, and Applications
From cluster ensemble to structure ensemble

Information Sciences: an International Journal
LoGID: An adaptive framework combining local and global incremental learning for dynamic selection of ensembles of HMMs

Pattern Recognition
A meta learning approach to grammatical error correction

ACL '12 Proceedings of the 50th Annual Meeting of the Association for Computational Linguistics: Short Papers - Volume 2
A novel automatic two-stage locally regularized classifier construction method using the extreme learning machine

Neurocomputing
Single Classifier-based Multiple Classification Scheme for weak classifiers: An experimental comparison

Expert Systems with Applications: An International Journal

Quantified Score

Hi-index	0.07

Visualization

Abstract

We discuss approaches to incrementally construct an ensemble. The first constructs an ensemble of classifiers choosing a subset from a larger set, and the second constructs an ensemble of discriminants, where a classifier is used for some classes only. We investigate criteria including accuracy, significant improvement, diversity, correlation, and the role of search direction. For discriminant ensembles, we test subset selection and trees. Fusion is by voting or by a linear model. Using 14 classifiers on 38 data sets, incremental search finds small, accurate ensembles in polynomial time. The discriminant ensemble uses a subset of discriminants and is simpler, interpretable, and accurate. We see that an incremental ensemble has higher accuracy than bagging and random subspace method; and it has a comparable accuracy to AdaBoost, but fewer classifiers.