Extremely randomized trees

Authors:
Pierre Geurts;Damien Ernst;Louis Wehenkel
Affiliations:
Department of Electrical Engineering and Computer Science, University of Liège, Liège, Belgium B-28, B-4000;Department of Electrical Engineering and Computer Science, University of Liège, Liège, Belgium B-28, B-4000;Department of Electrical Engineering and Computer Science, University of Liège, Liège, Belgium B-28, B-4000
Venue:
Machine Learning
Year:
2006

Citing 26
Cited 93

Decision trees and transient stability of electric power systems

Automatica (Journal of IFAC)
A Further Comparison of Splitting Rules for Decision-Tree Induction

Machine Learning
Neural networks and the bias/variance dilemma

Neural Computation
Original Contribution: Stacked generalization

Neural Networks
C4.5: programs for machine learning

C4.5: programs for machine learning
Bagging predictors

Machine Learning
Error reduction through learning multiple descriptions

Machine Learning
The Random Subspace Method for Constructing Decision Forests

IEEE Transactions on Pattern Analysis and Machine Intelligence
An Experimental Comparison of Three Methods for Constructing Ensembles of Decision Trees: Bagging, Boosting, and Randomization

Machine Learning
MultiBoosting: A Technique for Combining Boosting and Wagging

Machine Learning
Randomizing Outputs to Increase Prediction Accuracy

Machine Learning
Automatic Learning Techniques in Power Systems

Automatic Learning Techniques in Power Systems
Random Forests

Machine Learning
On Bias, Variance, 0/1—Loss, and the Curse-of-Dimensionality

Data Mining and Knowledge Discovery
An Empirical Comparison of Voting Classification Algorithms: Bagging, Boosting, and Variants

Machine Learning
An Empirical Comparison of Selection Measures for Decision-Tree Induction

Machine Learning
Variance and Bias for General Loss Functions

Machine Learning
Stochastic Attribute Selection Committees

AI '98 Selected papers from the 11th Australian Joint Conference on Artificial Intelligence on Advanced Topics in Artificial Intelligence
A decision-theoretic generalization of on-line learning and an application to boosting

EuroCOLT '95 Proceedings of the Second European Conference on Computational Learning Theory
Inference for the Generalization Error

Machine Learning
Bayes point machines

The Journal of Machine Learning Research
Multistrategy Ensemble Learning: Reducing Error by Combining Ensemble Learning Techniques

IEEE Transactions on Knowledge and Data Engineering
Bias-Variance Analysis of Support Vector Machines for the Development of SVM-Based Ensemble Methods

The Journal of Machine Learning Research
Proteomic mass spectra classification using decision tree based ensemble methods

Bioinformatics
Inductive learning of tree-based regression models[1]Available at http:www.ncc.up.pt/~ltorgo/PhD/.

AI Communications
Segment and combine approach for non-parametric time-series classification

PKDD'05 Proceedings of the 9th European conference on Principles and Practice of Knowledge Discovery in Databases

Kernelizing the output of tree-based methods

ICML '06 Proceedings of the 23rd international conference on Machine learning
Machine-learnt versus analytical models of TCP throughput

Computer Networks: The International Journal of Computer and Telecommunications Networking
Gradient boosting for kernelized output spaces

Proceedings of the 24th international conference on Machine learning
Forest-RK: A New Random Forest Induction Method

ICIC '08 Proceedings of the 4th international conference on Intelligent Computing: Advanced Intelligent Computing Theories and Applications - with Aspects of Artificial Intelligence
High scale video mining with forests of fuzzy decision trees

CSTST '08 Proceedings of the 5th international conference on Soft computing as transdisciplinary science and technology
Using decision trees to recognize visual events

AREA '08 Proceedings of the 1st ACM workshop on Analysis and retrieval of events/actions and workflows in video streams
Segmentation and Recognition Using Structure from Motion Point Clouds

ECCV '08 Proceedings of the 10th European Conference on Computer Vision: Part I
Random Subwindows for Robust Peak Recognition in Intracranial Pressure Signals

ISVC '08 Proceedings of the 4th International Symposium on Advances in Visual Computing
Frontal-view gait recognition by intra- and inter-frame rectangle size distribution

Pattern Recognition Letters
Probability Density Estimation by Perturbing and Combining Tree Structured Markov Networks

ECSQARU '09 Proceedings of the 10th European Conference on Symbolic and Quantitative Approaches to Reasoning with Uncertainty
A Bayesian Random Split to Build Ensembles of Classification Trees

ECSQARU '09 Proceedings of the 10th European Conference on Symbolic and Quantitative Approaches to Reasoning with Uncertainty
A Study of Random Linear Oracle Ensembles

MCS '09 Proceedings of the 8th International Workshop on Multiple Classifier Systems
Comparison of Bagging and Boosting Algorithms on Sample and Feature Weighting

MCS '09 Proceedings of the 8th International Workshop on Multiple Classifier Systems
Global Context Extraction for Object Recognition Using a Combination of Range and Visual Features

Dyn3D '09 Proceedings of the DAGM 2009 Workshop on Dynamic 3D Imaging
Learning with Few Examples by Transferring Feature Relevance

Proceedings of the 31st DAGM Symposium on Pattern Recognition
Adaptive treatment of epilepsy via batch-mode reinforcement learning

IAAI'08 Proceedings of the 20th national conference on Innovative applications of artificial intelligence - Volume 3
Spectrum of variable-random trees

Journal of Artificial Intelligence Research
Real-time planning for parameterized human motion

Proceedings of the 2008 ACM SIGGRAPH/Eurographics Symposium on Computer Animation
Real-time bag of words, approximately

Proceedings of the ACM International Conference on Image and Video Retrieval
Dense sampling and fast encoding for 3D model retrieval using bag-of-visual features

Proceedings of the ACM International Conference on Image and Video Retrieval
Reinforcement learning versus model predictive control: a comparison on a power system problem

IEEE Transactions on Systems, Man, and Cybernetics, Part B: Cybernetics
Lossless Online Ensemble Learning (LOEL) and Its Application to Subcortical Segmentation

MICCAI '09 Proceedings of the 12th International Conference on Medical Image Computing and Computer-Assisted Intervention: Part II
Object Recognition Based on Efficient Sub-window Search

AICI '09 Proceedings of the International Conference on Artificial Intelligence and Computational Intelligence
Randomized Tree Ensembles for Object Detection in Computational Pathology

ISVC '09 Proceedings of the 5th International Symposium on Advances in Visual Computing: Part I
On the selection of decision trees in random forests

IJCNN'09 Proceedings of the 2009 international joint conference on Neural Networks
A Study of Parts-Based Object Class Detection Using Complete Graphs

International Journal of Computer Vision
Why fuzzy decision trees are good rankers

IEEE Transactions on Fuzzy Systems
Unsupervised human motion analysis using automatic label trees

SMC'09 Proceedings of the 2009 IEEE international conference on Systems, Man and Cybernetics
Incremental indexing and distributed image search using shared randomized vocabularies

Proceedings of the international conference on Multimedia information retrieval
Category Level Object Segmentation by Combining Bag-of-Words Models with Dirichlet Processes and Random Fields

International Journal of Computer Vision
Ensemble classification based on generalized additive models

Computational Statistics & Data Analysis
Content-based image retrieval by indexing random subwindows with randomized trees

ACCV'07 Proceedings of the 8th Asian conference on Computer vision - Volume Part II
Towards a better understanding of random forests through the study of strength and correlation

ICIC'09 Proceedings of the Intelligent computing 5th international conference on Emerging intelligent computing technology and applications
An incremental extremely random forest classifier for online learning and tracking

ICIP'09 Proceedings of the 16th IEEE international conference on Image processing
Visual event recognition using decision trees

Multimedia Tools and Applications
Combining predictions for accurate recommender systems

Proceedings of the 16th ACM SIGKDD international conference on Knowledge discovery and data mining
Polarimetric SAR image classification using multifeatures combination and extremely randomized clustering forests

EURASIP Journal on Advances in Signal Processing - Special issue on advances in multidimensional synthetic aperture radar signal processing
Distance metric learning and feature combination for shape-based 3D model retrieval

Proceedings of the ACM workshop on 3D object retrieval
Weakly supervised classification of objects in images using soft random forests

ECCV'10 Proceedings of the 11th European conference on Computer vision: Part IV
Semantic segmentation of urban scenes using dense depth maps

ECCV'10 Proceedings of the 11th European conference on Computer vision: Part IV
Learning with ensembles of randomized trees: new insights

ECML PKDD'10 Proceedings of the 2010 European conference on Machine learning and knowledge discovery in databases: Part III
Learning with few examples for binary and multiclass classification using regularization of randomized trees

Pattern Recognition Letters
Randomized locality sensitive vocabularies for bag-of-features model

ECCV'10 Proceedings of the 11th European conference on computer vision conference on Computer vision: Part III
Inference on the prediction of ensembles of infinite size

Pattern Recognition
Unsupervised moving object detection with on-line generalized hough transform

ACCV'10 Proceedings of the 10th Asian conference on Computer vision - Volume Part III
Randomised manifold forests for principal angle-based face recognition

ACCV'10 Proceedings of the 10th Asian conference on Computer vision - Volume Part IV
Efficient Gaussian process classification using random decision forests

Pattern Recognition and Image Analysis
Entangled decision forests and their application for semantic segmentation of CT images

IPMI'11 Proceedings of the 22nd international conference on Information processing in medical imaging
On oblique random forests

ECML PKDD'11 Proceedings of the 2011 European conference on Machine learning and knowledge discovery in databases - Volume Part II
Estimation of human orientation in images captured with a range camera

ACIVS'11 Proceedings of the 13th international conference on Advanced concepts for intelligent vision systems
New object detection features in the OpenCV library

Pattern Recognition and Image Analysis
GRASP forest: a new ensemble method for trees

MCS'11 Proceedings of the 10th international conference on Multiple classifier systems
Improving the classification accuracy of the classic RF method by intelligent feature selection and weighted voting of trees with application to medical image segmentation

MLMI'11 Proceedings of the Second international conference on Machine learning in medical imaging
Regionwise classification of building facade images

PIA'11 Proceedings of the 2011 ISPRS conference on Photogrammetric image analysis
Automatic localization of interest points in zebrafish images with tree-based methods

PRIB'11 Proceedings of the 6th IAPR international conference on Pattern recognition in bioinformatics
Approximate policy iteration for closed-loop learning of visual tasks

ECML'06 Proceedings of the 17th European conference on Machine Learning
Robust analysis of silhouettes by morphological size distributions

ACIVS'06 Proceedings of the 8th international conference on Advanced Concepts For Intelligent Vision Systems
The Visual Extent of an Object

International Journal of Computer Vision
Novel ensemble methods for regression via classification problems

Expert Systems with Applications: An International Journal
Co-occurrence random forests for object localization and classification

ACCV'09 Proceedings of the 9th Asian conference on Computer Vision - Volume Part III
Reinforcement learning with raw image pixels as input state

IWICPAS'06 Proceedings of the 2006 Advances in Machine Vision, Image Processing, and Pattern Analysis international conference on Intelligent Computing in Pattern Analysis/Synthesis
Data-driven dynamic emulation modelling for the optimal management of environmental systems

Environmental Modelling & Software
Decision Forests: A Unified Framework for Classification, Regression, Density Estimation, Manifold Learning and Semi-Supervised Learning

Foundations and Trends® in Computer Graphics and Vision
Building implicit dictionaries based on extreme random clustering for modality recognition

MCBR-CDS'11 Proceedings of the Second MICCAI international conference on Medical Content-Based Retrieval for Clinical Decision Support
Large-scale Gaussian process classification using random decision forests

Pattern Recognition and Image Analysis
Dynamic Random Forests

Pattern Recognition Letters
Goal-Directed online learning of predictive models

EWRL'11 Proceedings of the 9th European conference on Recent Advances in Reinforcement Learning
An automated vision based on-line novel percept detection method for a mobile robot

Robotics and Autonomous Systems
SHREC'10 track: large scale retrieval

EG 3DOR'10 Proceedings of the 3rd Eurographics conference on 3D Object Retrieval
SHREC'10 track: generic 3D warehouse

EG 3DOR'10 Proceedings of the 3rd Eurographics conference on 3D Object Retrieval
SHREC'10 track: non-rigid 3D shape retrieval

EG 3DOR'10 Proceedings of the 3rd Eurographics conference on 3D Object Retrieval
SHREC'10 track: range scan retrieval

EG 3DOR'10 Proceedings of the 3rd Eurographics conference on 3D Object Retrieval
3D object retrieval via range image queries based on SIFT descriptors on panoramic views

EG 3DOR'12 Proceedings of the 5th Eurographics conference on 3D Object Retrieval
Texture based decision tree classification for Arecanut

Proceedings of the CUBE International Information Technology Conference
Stratified sampling for feature subspace selection in random forests for high dimensional data

Pattern Recognition
Annotation propagation in large image databases via dense image correspondence

ECCV'12 Proceedings of the 12th European conference on Computer Vision - Volume Part III
People orientation recognition by mixtures of wrapped distributions on random trees

ECCV'12 Proceedings of the 12th European conference on Computer Vision - Volume Part V
Embedding monte carlo search of features in tree-based ensemble methods

ECML PKDD'12 Proceedings of the 2012 European conference on Machine Learning and Knowledge Discovery in Databases - Volume Part I
Ensembles on random patches

ECML PKDD'12 Proceedings of the 2012 European conference on Machine Learning and Knowledge Discovery in Databases - Volume Part I
Learning motion controllers with adaptive depth perception

EUROSCA'12 Proceedings of the 11th ACM SIGGRAPH / Eurographics conference on Computer Animation
Learning motion controllers with adaptive depth perception

Proceedings of the ACM SIGGRAPH/Eurographics Symposium on Computer Animation
A new random forest method for one-class classification

SSPR'12/SPR'12 Proceedings of the 2012 Joint IAPR international conference on Structural, Syntactic, and Statistical Pattern Recognition
How large should ensembles of classifiers be?

Pattern Recognition
On the prediction of popularity of trends and hits for user generated videos

Proceedings of the sixth ACM international conference on Web search and data mining
Relative forest for attribute prediction

ACCV'12 Proceedings of the 11th Asian conference on Computer Vision - Volume Part I
One class random forests

Pattern Recognition
Hough-based tracking of non-rigid objects

Computer Vision and Image Understanding
A multi modal approach to gesture recognition from audio and video data

Proceedings of the 15th ACM on International conference on multimodal interaction
Detecting bipedal motion from correlated probabilistic trajectories

Pattern Recognition Letters
Dynamic random regression forests for real-time head pose estimation

Machine Vision and Applications
Effective named entity recognition for idiosyncratic web collections

Proceedings of the 23rd international conference on World wide web
A comparison of methods for sketch-based 3D shape retrieval

Computer Vision and Image Understanding
3D object retrieval via range image queries in a bag-of-visual-words context

The Visual Computer: International Journal of Computer Graphics

Quantified Score

Hi-index	0.00

Visualization

Abstract

This paper proposes a new tree-based ensemble method for supervised classification and regression problems. It essentially consists of randomizing strongly both attribute and cut-point choice while splitting a tree node. In the extreme case, it builds totally randomized trees whose structures are independent of the output values of the learning sample. The strength of the randomization can be tuned to problem specifics by the appropriate choice of a parameter. We evaluate the robustness of the default choice of this parameter, and we also provide insight on how to adjust it in particular situations. Besides accuracy, the main strength of the resulting algorithm is computational efficiency. A bias/variance analysis of the Extra-Trees algorithm is also provided as well as a geometrical and a kernel characterization of the models induced.