Towards a better understanding of random forests through the study of strength and correlation

Authors:
Simon Bernard;Laurent Heutte;Sébastien Adam
Affiliations:
Université de Rouen, LITIS, Saint-Etienne du Rouvray, France;Université de Rouen, LITIS, Saint-Etienne du Rouvray, France;Université de Rouen, LITIS, Saint-Etienne du Rouvray, France
Venue:
ICIC'09 Proceedings of the Intelligent computing 5th international conference on Emerging intelligent computing technology and applications
Year:
2009

Citing 8
Cited 0

Bagging predictors

Machine Learning
The Random Subspace Method for Constructing Decision Forests

IEEE Transactions on Pattern Analysis and Machine Intelligence
Approximate statistical tests for comparing supervised classification learning algorithms

Neural Computation
Random Forests

Machine Learning
Extremely randomized trees

Machine Learning
Rotation Forest: A New Classifier Ensemble Method

IEEE Transactions on Pattern Analysis and Machine Intelligence
A two-stage outlier rejection strategy for numerical field extraction in handwritten documents

ICPR '06 Proceedings of the 18th International Conference on Pattern Recognition - Volume 03
Using Random Forests for Handwritten Digit Recognition

ICDAR '07 Proceedings of the Ninth International Conference on Document Analysis and Recognition - Volume 02

Quantified Score

Hi-index	0.00

Visualization

Abstract

In this paper we present a study on the Random Forest (RF) family of ensemble methods. From our point of view, a "classical" RF induction process presents two main drawbacks : (i) the number of trees has to be a priori fixed (ii) trees are independently, thus arbitrarily, added to the ensemble due to the randomization. Hence, this kind of process offers no guarantee that all the trees will well cooperate into the same committee. In this work we thus propose to study the RF mechanisms that explain this cooperation by analysing, for particular subsets of trees called sub-forests, the link between accuracy and properties such as Strength and Correlation. We show that these properties, through the Correlation/Strengh2 ratio, should be taken into account to explain the sub-forest performance.