Multi-information ensemble diversity

Authors:
Zhi-Hua Zhou;Nan Li
Affiliations:
National Key Laboratory for Novel Software Technology, Nanjing University, Nanjing, China;School of Mathematical Sciences, Soochow University, Suzhou, China
Venue:
MCS'10 Proceedings of the 9th international conference on Multiple Classifier Systems
Year:
2010

Citing 8
Cited 4

Elements of information theory

Elements of information theory
The Random Subspace Method for Constructing Decision Forests

IEEE Transactions on Pattern Analysis and Machine Intelligence
The multiinformation function as a tool for measuring stachastic dependence

Learning in graphical models
An Experimental Comparison of Three Methods for Constructing Ensembles of Decision Trees: Bagging, Boosting, and Randomization

Machine Learning
Measures of Diversity in Classifier Ensembles and Their Relationship with the Ensemble Accuracy

Machine Learning
Multivariate information bottleneck

Neural Computation
An Information Theoretic Perspective on Multiple Classifier Systems

MCS '09 Proceedings of the 8th International Workshop on Multiple Classifier Systems
Information theoretical analysis of multivariate correlation

IBM Journal of Research and Development

Normalized joint mutual information measure for image segmentation evaluation with multiple ground-truth images

CAIP'11 Proceedings of the 14th international conference on Computer analysis of images and patterns - Volume Part I
The impact of diversity on the accuracy of evidential classifier ensembles

International Journal of Approximate Reasoning
Diversity regularized machine

IJCAI'11 Proceedings of the Twenty-Second international joint conference on Artificial Intelligence - Volume Volume Two
Diversity regularized ensemble pruning

ECML PKDD'12 Proceedings of the 2012 European conference on Machine Learning and Knowledge Discovery in Databases - Volume Part I

Quantified Score

Hi-index	0.00

Visualization

Abstract

Understanding ensemble diversity is one of the most important fundamental issues in ensemble learning. Inspired by a recent work trying to explain ensemble diversity from the information theoretic perspective, in this paper we study the ensemble diversity from the view of multi-information. We show that from this view, the ensemble diversity can be decomposed over the component classifiers constituting the ensemble. Based on this formulation, an approximation is given for estimating the diversity in practice. Experimental results show that our formulation and approximation are promising.