Decision forests with oblique decision trees

Authors:
Peter J. Tan;David L. Dowe
Affiliations:
School of Computer Science and Software Engineering, Monash University, Clayton, Australia;School of Computer Science and Software Engineering, Monash University, Clayton, Australia
Venue:
MICAI'06 Proceedings of the 5th Mexican international conference on Artificial Intelligence
Year:
2006

Citing 12
Cited 3

C4.5: programs for machine learning

C4.5: programs for machine learning
Bagging predictors

Machine Learning
The Random Subspace Method for Constructing Decision Forests

IEEE Transactions on Pattern Analysis and Machine Intelligence
An Experimental Comparison of Three Methods for Constructing Ensembles of Decision Trees: Bagging, Boosting, and Randomization

Machine Learning
Randomizing Outputs to Increase Prediction Accuracy

Machine Learning
Random Forests

Machine Learning
MML Inference of Decision Graphs with Multi-way Joins

AI '02 Proceedings of the 15th Australian Joint Conference on Artificial Intelligence: Advances in Artificial Intelligence
Delegating classifiers

ICML '04 Proceedings of the twenty-first international conference on Machine learning
Statistical and Inductive Inference by Minimum Message Length (Information Science and Statistics)

Statistical and Inductive Inference by Minimum Message Length (Information Science and Statistics)
Data Mining: Practical Machine Learning Tools and Techniques, Second Edition (Morgan Kaufmann Series in Data Management Systems)

Data Mining: Practical Machine Learning Tools and Techniques, Second Edition (Morgan Kaufmann Series in Data Management Systems)
Statistical Comparisons of Classifiers over Multiple Data Sets

The Journal of Machine Learning Research
MML inference of oblique decision trees

AI'04 Proceedings of the 17th Australian joint conference on Advances in Artificial Intelligence

Building classification models from microarray data with tree-based classification algorithms

AI'07 Proceedings of the 20th Australian joint conference on Advances in artificial intelligence
Implementation of a scalable decision forest model based on information theory

Expert Systems with Applications: An International Journal
On oblique random forests

ECML PKDD'11 Proceedings of the 2011 European conference on Machine learning and knowledge discovery in databases - Volume Part II

Quantified Score

Hi-index	0.00

Visualization

Abstract

Ensemble learning schemes have shown impressive increases in prediction accuracy over single model schemes. We introduce a new decision forest learning scheme, whose base learners are Minimum Message Length (MML) oblique decision trees. Unlike other tree inference algorithms, MML oblique decision tree learning does not over-grow the inferred trees. The resultant trees thus tend to be shallow and do not require pruning. MML decision trees are known to be resistant to over-fitting and excellent at probabilistic predictions. A novel weighted averaging scheme is also proposed which takes advantage of high probabilistic prediction accuracy produced by MML oblique decision trees. The experimental results show that the new weighted averaging offers solid improvement over other averaging schemes, such as majority vote. Our MML decision forests scheme also returns favourable results compared to other ensemble learning algorithms on data sets with binary classes.