Ensemble selection from libraries of models

Authors:
Rich Caruana;Alexandru Niculescu-Mizil;Geoff Crew;Alex Ksikes
Affiliations:
Cornell University, Ithaca, NY;Cornell University, Ithaca, NY;Cornell University, Ithaca, NY;Cornell University, Ithaca, NY
Venue:
ICML '04 Proceedings of the twenty-first international conference on Machine learning
Year:
2004

Citing 8
Cited 62

Original Contribution: Stacked generalization

Neural Networks
Bagging predictors

Machine Learning
Wrappers for feature subset selection

Artificial Intelligence - Special issue on relevance
Feature selection for ensembles

AAAI '99/IAAI '99 Proceedings of the sixteenth national conference on Artificial intelligence and the eleventh Innovative applications of artificial intelligence conference innovative applications of artificial intelligence
Bayesian Averaging of Classifiers and the Overfitting Problem

ICML '00 Proceedings of the Seventeenth International Conference on Machine Learning
FeatureBoost: A Meta-Learning Algorithm that Improves Model Robustness

ICML '00 Proceedings of the Seventeenth International Conference on Machine Learning
Tree Induction for Probability-Based Ranking

Machine Learning
Solving multiclass learning problems via error-correcting output codes

Journal of Artificial Intelligence Research

KDD-Cup 2004: results and analysis

ACM SIGKDD Explorations Newsletter
Sharing Classifiers among Ensembles from Related Problem Domains

ICDM '05 Proceedings of the Fifth IEEE International Conference on Data Mining
Q2C@UST: our winning solution to query classification in KDDCUP 2005

ACM SIGKDD Explorations Newsletter
Model compression

Proceedings of the 12th ACM SIGKDD international conference on Knowledge discovery and data mining
Query enrichment for web-query classification

ACM Transactions on Information Systems (TOIS)
Optimizing to arbitrary NLP metrics using ensemble selection

HLT '05 Proceedings of the conference on Human Language Technology and Empirical Methods in Natural Language Processing
Ensemble Pruning Via Semi-definite Programming

The Journal of Machine Learning Research
Comparisons of sequence labeling algorithms and extensions

Proceedings of the 24th international conference on Machine learning
A support vector method for optimizing average precision

SIGIR '07 Proceedings of the 30th annual international ACM SIGIR conference on Research and development in information retrieval
Model-shared subspace boosting for multi-label classification

Proceedings of the 13th ACM SIGKDD international conference on Knowledge discovery and data mining
Semantics reinforcement and fusion learning for multimedia streams

Proceedings of the 6th ACM international conference on Image and video retrieval
To Select or To Weigh: A Comparative Study of Linear Combination Schemes for SuperParent-One-Dependence Estimators

IEEE Transactions on Knowledge and Data Engineering
Greedy regression ensemble selection: Theory and an application to water quality prediction

Information Sciences: an International Journal
Resampling-based selective clustering ensembles

Pattern Recognition Letters
Collective-agreement-based pruning of ensembles

Computational Statistics & Data Analysis
Classification models for the prediction of clinicians' information needs

Journal of Biomedical Informatics
Incremental construction of classifier and discriminant ensembles

Information Sciences: an International Journal
Pruning an ensemble of classifiers via reinforcement learning

Neurocomputing
Focused Ensemble Selection: A Diversity-Based Method for Greedy Ensemble Selection

Proceedings of the 2008 conference on ECAI 2008: 18th European Conference on Artificial Intelligence
MLP, Gaussian Processes and Negative Correlation Learning for Time Series Prediction

MCS '09 Proceedings of the 8th International Workshop on Multiple Classifier Systems
Ensemble Learning: A Study on Different Variants of the Dynamic Selection Approach

MLDM '09 Proceedings of the 6th International Conference on Machine Learning and Data Mining in Pattern Recognition
Taxonomy for characterizing ensemble methods in classification tasks: A review and annotated bibliography

Computational Statistics & Data Analysis
A fast ensemble pruning algorithm based on pattern mining process

Data Mining and Knowledge Discovery
Managing domain knowledge and multiple models with boosting

IJCAI'07 Proceedings of the 20th international joint conference on Artifical intelligence
Large-scale multimedia semantic concept modeling using robust subspace bagging and MapReduce

LS-MMRM '09 Proceedings of the First ACM workshop on Large-scale multimedia retrieval and mining
A multi-model selection framework for unknown and/or evolutive misclassification cost problems

Pattern Recognition
Evaluation of robustness and performance of early stopping rules with multi layer perceptrons

IJCNN'09 Proceedings of the 2009 international joint conference on Neural Networks
Ensemble-based classifiers

Artificial Intelligence Review
Design and analysis of the KDD cup 2009: fast scoring on a large orange customer database

ACM SIGKDD Explorations Newsletter
Combining predictions for accurate recommender systems

Proceedings of the 16th ACM SIGKDD international conference on Knowledge discovery and data mining
Combining multiple classification or regression models using genetic algorithms

RSCTC'10 Proceedings of the 7th international conference on Rough sets and current trends in computing
Web spam classification: a few features worth more

Proceedings of the 2011 Joint WICOW/AIRWeb Workshop on Web Quality
Empirical analysis and evaluation of approximate techniques for pruning regression bagging ensembles

Neurocomputing
Ensembling predictions of student knowledge within intelligent tutoring systems

UMAP'11 Proceedings of the 19th international conference on User modeling, adaption, and personalization
Adaptive ROC-based ensembles of HMMs applied to anomaly detection

Pattern Recognition
Ensemble pruning via base-classifier replacement

WAIM'11 Proceedings of the 12th international conference on Web-age information management
A new metric for greedy ensemble pruning

AICI'11 Proceedings of the Third international conference on Artificial intelligence and computational intelligence - Volume Part II
Ensemble pruning using reinforcement learning

SETN'06 Proceedings of the 4th Helenic conference on Advances in Artificial Intelligence
Eigenclassifiers for combining correlated classifiers

Information Sciences: an International Journal
Comparative analysis of learning and meta-learning algorithms for creating models for predicting the probable alcohol level during the ripening of grape berries

Computers and Electronics in Agriculture
On selecting additional predictive models in double bagging type ensemble method

ICCSA'10 Proceedings of the 2010 international conference on Computational Science and Its Applications - Volume Part IV
Selective voting in convex-hull ensembles improves classification accuracy

Artificial Intelligence in Medicine
A double pruning algorithm for classification ensembles

MCS'10 Proceedings of the 9th international conference on Multiple Classifier Systems
Multivariate statistical tests for comparing classification algorithms

LION'05 Proceedings of the 5th international conference on Learning and Intelligent Optimization
Bagging ensemble selection

AI'11 Proceedings of the 24th international conference on Advances in Artificial Intelligence
Ensemble pruning for text categorization based on data partitioning

AIRS'11 Proceedings of the 7th Asia conference on Information Retrieval Technology
The sum is greater than the parts: ensembling models of student knowledge in educational software

ACM SIGKDD Explorations Newsletter
Expert pruning based on genetic algorithm in regression problems

ACIIDS'12 Proceedings of the 4th Asian conference on Intelligent Information and Database Systems - Volume Part III
Ensemble pruning using harmony search

HAIS'12 Proceedings of the 7th international conference on Hybrid Artificial Intelligent Systems - Volume Part II
Tapprints: your finger taps have fingerprints

Proceedings of the 10th international conference on Mobile systems, applications, and services
Ensemble approaches for regression: A survey

ACM Computing Surveys (CSUR)
A competitive ensemble pruning approach based on cross-validation technique

Knowledge-Based Systems
Diversity regularized ensemble pruning

ECML PKDD'12 Proceedings of the 2012 European conference on Machine Learning and Knowledge Discovery in Databases - Volume Part I
A class centric feature and classifier ensemble selection approach for music genre classification

SSPR'12/SPR'12 Proceedings of the 2012 Joint IAPR international conference on Structural, Syntactic, and Statistical Pattern Recognition
Bagging ensemble selection for regression

AI'12 Proceedings of the 25th Australasian joint conference on Advances in Artificial Intelligence
Intrusion detection using disagreement-based semi-supervised learning: detection enhancement and false alarm reduction

CSS'12 Proceedings of the 4th international conference on Cyberspace Safety and Security
Embedding change rate estimation based on ensemble learning

Proceedings of the first ACM workshop on Information hiding and multimedia security
Malware detection by pruning of parallel ensembles using harmony search

Pattern Recognition Letters
Comparison of a genetic algorithm and simulated annealing for automatic neural network ensemble development

Neurocomputing
A novel ensemble pruning algorithm based on randomized greedy selective strategy and ballot

Neurocomputing
A metric for unsupervised metalearning

Intelligent Data Analysis
Combining multiple predictive models using genetic algorithms

Intelligent Data Analysis - Combined Learning Methods and Mining Complex Data

Quantified Score

Hi-index	0.00

Visualization

Abstract

We present a method for constructing ensembles from libraries of thousands of models. Model libraries are generated using different learning algorithms and parameter settings. Forward stepwise selection is used to add to the ensemble the models that maximize its performance. Ensemble selection allows ensembles to be optimized to performance metric such as accuracy, cross entropy, mean precision, or ROC Area. Experiments with seven test problems and ten metrics demonstrate the benefit of ensemble selection.