Diversity regularized ensemble pruning

Authors:
Nan Li;Yang Yu;Zhi-Hua Zhou
Affiliations:
National Key Laboratory for Novel Software Technology, Nanjing University, Nanjing, China, School of Mathematical Sciences, Soochow University, Suzhou, China;National Key Laboratory for Novel Software Technology, Nanjing University, Nanjing, China;National Key Laboratory for Novel Software Technology, Nanjing University, Nanjing, China
Venue:
ECML PKDD'12 Proceedings of the 2012 European conference on Machine Learning and Knowledge Discovery in Databases - Volume Part I
Year:
2012

Citing 24
Cited 3

A theory of the learnable

Communications of the ACM
Bagging predictors

Machine Learning
An Experimental Comparison of Three Methods for Constructing Ensembles of Decision Trees: Bagging, Boosting, and Randomization

Machine Learning
Ensembling neural networks: many could be better than all

Artificial Intelligence
Measures of Diversity in Classifier Ensembles and Their Relationship with the Ensemble Accuracy

Machine Learning
On the Boosting Pruning Problem

ECML '00 Proceedings of the 11th European Conference on Machine Learning
Pruning Adaptive Boosting

ICML '97 Proceedings of the Fourteenth International Conference on Machine Learning
Covering number bounds of certain regularized linear function classes

The Journal of Machine Learning Research
Ensemble selection from libraries of models

ICML '04 Proceedings of the twenty-first international conference on Machine learning
A Theoretical and Experimental Analysis of Linear Combiners for Multiple Classifier Systems

IEEE Transactions on Pattern Analysis and Machine Intelligence
Pruning in ordered bagging ensembles

ICML '06 Proceedings of the 23rd international conference on Machine learning
An analysis of diversity measures

Machine Learning
Statistical Comparisons of Classifiers over Multiple Data Sets

The Journal of Machine Learning Research
Ensemble Pruning Via Semi-definite Programming

The Journal of Machine Learning Research
An Analysis of Ensemble Pruning Techniques Based on Ordered Aggregation

IEEE Transactions on Pattern Analysis and Machine Intelligence
Focused Ensemble Selection: A Diversity-Based Method for Greedy Ensemble Selection

Proceedings of the 2008 conference on ECAI 2008: 18th European Conference on Artificial Intelligence
An Information Theoretic Perspective on Multiple Classifier Systems

MCS '09 Proceedings of the 8th International Workshop on Multiple Classifier Systems
Selective Ensemble under Regularization Framework

MCS '09 Proceedings of the 8th International Workshop on Multiple Classifier Systems
Predictive Ensemble Pruning by Expectation Propagation

IEEE Transactions on Knowledge and Data Engineering
Empirical analysis and evaluation of approximate techniques for pruning regression bagging ensembles

Neurocomputing
Multi-information ensemble diversity

MCS'10 Proceedings of the 9th international conference on Multiple Classifier Systems
The sample complexity of pattern classification with neural networks: the size of the weights is more important than the size of the network

IEEE Transactions on Information Theory
Diversity regularized machine

IJCAI'11 Proceedings of the Twenty-Second international joint conference on Artificial Intelligence - Volume Volume Two
Ensemble Methods: Foundations and Algorithms

Ensemble Methods: Foundations and Algorithms

Classifier ensemble using a heuristic learning with sparsity and diversity

ICONIP'12 Proceedings of the 19th international conference on Neural Information Processing - Volume Part II
Embedding change rate estimation based on ensemble learning

Proceedings of the first ACM workshop on Information hiding and multimedia security
A novel classifier ensemble method with sparsity and diversity

Neurocomputing

Quantified Score

Hi-index	0.00

Visualization

Abstract

Diversity among individual classifiers is recognized to play a key role in ensemble, however, few theoretical properties are known for classification. In this paper, by focusing on the popular ensemble pruning setting (i.e., combining classifier by voting and measuring diversity in pairwise manner), we present a theoretical study on the effect of diversity on the generalization performance of voting in the PAC-learning framework. It is disclosed that the diversity is closely-related to the hypothesis space complexity, and encouraging diversity can be regarded to apply regularization on ensemble methods. Guided by this analysis, we apply explicit diversity regularization to ensemble pruning, and propose the Diversity Regularized Ensemble Pruning (DREP) method. Experimental results show the effectiveness of DREP.