Decision trees using model ensemble-based nodes

Authors:
Hakan Altınçay
Affiliations:
Department of Computer Engineering, Eastern Mediterranean University, Mağusa, Northern Cyprus, Turkey
Venue:
Pattern Recognition
Year:
2007

Citing 23
Cited 5

An incremental method for finding multivariate splits for decision trees

Proceedings of the seventh international conference (1990) on Machine learning
Neural networks and the bias/variance dilemma

Neural Computation
Multivariate Decision Trees

Machine Learning
Bagging predictors

Machine Learning
A Comparative Analysis of Methods for Pruning Decision Trees

IEEE Transactions on Pattern Analysis and Machine Intelligence
On Combining Classifiers

IEEE Transactions on Pattern Analysis and Machine Intelligence
The Random Subspace Method for Constructing Decision Forests

IEEE Transactions on Pattern Analysis and Machine Intelligence
Combined 5 × 2 cv F test for comparing supervised classification learning algorithms

Neural Computation
An information theoretic framework for weight estimation in the combination of probabilistic classifiers for speaker identification

Speech Communication
Induction of Decision Trees

Machine Learning
Dynamic Automatic Model Selection

Dynamic Automatic Model Selection
Pattern Classification (2nd Edition)

Pattern Classification (2nd Edition)
Combining Pattern Classifiers: Methods and Algorithms

Combining Pattern Classifiers: Methods and Algorithms
A system for induction of oblique decision trees

Journal of Artificial Intelligence Research
Constructing diverse classifier ensembles using artificial training examples

IJCAI'03 Proceedings of the 18th international joint conference on Artificial intelligence
Neural networks for classification: a survey

IEEE Transactions on Systems, Man, and Cybernetics, Part C: Applications and Reviews
Top-down induction of decision trees classifiers - a survey

IEEE Transactions on Systems, Man, and Cybernetics, Part C: Applications and Reviews
Ensembling local learners ThroughMultimodal perturbation

IEEE Transactions on Systems, Man, and Cybernetics, Part B: Cybernetics
Omnivariate decision trees

IEEE Transactions on Neural Networks
Perceptron-based learning algorithms

IEEE Transactions on Neural Networks
Classifiability-based omnivariate decision trees

IEEE Transactions on Neural Networks
Classification trees with neural network feature extraction

IEEE Transactions on Neural Networks
An improved algorithm for neural network classification of imbalanced training sets

IEEE Transactions on Neural Networks

Applying enhanced data mining approaches in predicting bank performance: A case of Taiwanese commercial banks

Expert Systems with Applications: An International Journal
Taxonomy for characterizing ensemble methods in classification tasks: A review and annotated bibliography

Computational Statistics & Data Analysis
Model selection in omnivariate decision trees using Structural Risk Minimization

Information Sciences: an International Journal
Decision trees: a recent overview

Artificial Intelligence Review
Building fast decision trees from large training sets

Intelligent Data Analysis

Quantified Score

Hi-index	0.03

Visualization

Abstract

Decision trees recursively partition the instance space by generating nodes that implement a decision function belonging to an a priori specified model class. Each decision may be univariate, linear or nonlinear. Alternatively, in omnivariate decision trees, one of the model types is dynamically selected by taking into account the complexity of the problem defined by the samples reaching that node. The selection is based on statistical tests where the most appropriate model type is selected as the one providing significantly better accuracy than others. In this study, we propose the use of model ensemble-based nodes where a multitude of models are considered for making decisions at each node. The ensemble members are generated by perturbing the model parameters and input attributes. Experiments conducted on several datasets and three model types indicate that the proposed approach achieves better classification accuracies compared to individual nodes, even in cases when only one model class is used in generating ensemble members.