Ensembles of Multi-Objective Decision Trees

Authors:
Dragi Kocev;Celine Vens;Jan Struyf;Sašo Džeroski
Affiliations:
Department of Knowledge Technologies, Jožef Stefan Institute, Jamova 39, 1000 Ljubljana, Slovenia;Department of Computer Science, Katholieke Universiteit Leuven, Celestijnenlaan 200A, 3001 Leuven, Belgium;Department of Computer Science, Katholieke Universiteit Leuven, Celestijnenlaan 200A, 3001 Leuven, Belgium;Department of Knowledge Technologies, Jožef Stefan Institute, Jamova 39, 1000 Ljubljana, Slovenia
Venue:
ECML '07 Proceedings of the 18th European conference on Machine Learning
Year:
2007

Citing 18
Cited 12

Decision Combination in Multiple Classifier Systems

IEEE Transactions on Pattern Analysis and Machine Intelligence
Bagging predictors

Machine Learning
First Order Regression

Machine Learning - special issue on inductive logic programming
Multitask Learning

Machine Learning - Special issue on inductive transfer
On Combining Classifiers

IEEE Transactions on Pattern Analysis and Machine Intelligence
Random Forests

Machine Learning
Predicting Chemical Parameters of River Water Quality from Bioindicator Data

Applied Intelligence
An Empirical Comparison of Voting Classification Algorithms: Bagging, Boosting, and Variants

Machine Learning
Neural Network Ensembles

IEEE Transactions on Pattern Analysis and Machine Intelligence
Top-Down Induction of Clustering Trees

ICML '98 Proceedings of the Fifteenth International Conference on Machine Learning
Simultaneous Prediction of Mulriple Chemical Parameters of River Water Quality with TILDE

PKDD '99 Proceedings of the Third European Conference on Principles of Data Mining and Knowledge Discovery
Bloomy Decision Tree for Multi-objective Classification

PKDD '01 Proceedings of the 5th European Conference on Principles of Data Mining and Knowledge Discovery
Knowledge Discovery in Multi-label Phenotype Data

PKDD '01 Proceedings of the 5th European Conference on Principles of Data Mining and Knowledge Discovery
Ensemble Methods in Machine Learning

MCS '00 Proceedings of the First International Workshop on Multiple Classifier Systems
A Comparison of Decision Tree Ensemble Creation Techniques

IEEE Transactions on Pattern Analysis and Machine Intelligence
Beam search induction and similarity constraints for predictive clustering trees

KDID'06 Proceedings of the 5th international conference on Knowledge discovery in inductive databases
Decision trees for hierarchical multilabel classification: a case study in functional genomics

PKDD'06 Proceedings of the 10th European conference on Principle and Practice of Knowledge Discovery in Databases
Constraint based induction of multi-objective regression trees

KDID'05 Proceedings of the 4th international conference on Knowledge Discovery in Inductive Databases

ImageCLEF 2009 medical image annotation task: PCTs for hierarchical multi-label classification

CLEF'09 Proceedings of the 10th international conference on Cross-language evaluation forum: multimedia experiments
Evaluation of distance measures for multi-class classification in binary SVM decision tree

ICAISC'10 Proceedings of the 10th international conference on Artificial intelligence and soft computing: Part I
Detection of visual concepts and annotation of images using ensembles of trees for hierarchical multi-label classification

ICPR'10 Proceedings of the 20th International conference on Recognizing patterns in signals, speech, images, and videos
Hierarchical annotation of medical images

Pattern Recognition
An extensive experimental comparison of methods for multi-label learning

Pattern Recognition
Tree ensembles for predicting structured outputs

Pattern Recognition
Multi-Label Classification Method for Multimedia Tagging

International Journal of Multimedia Data Engineering & Management
Iterative classification for multiple target attributes

Journal of Intelligent Information Systems
Context-Aware predictions on business processes: an ensemble-based solution

NFMCP'12 Proceedings of the First international conference on New Frontiers in Mining Complex Patterns
Multi-label learning with millions of labels: recommending advertiser bid phrases for web pages

Proceedings of the 22nd international conference on World Wide Web
Multi-target regression with rule ensembles

The Journal of Machine Learning Research
Efficient monte carlo methods for multi-dimensional learning with classifier chains

Pattern Recognition

Quantified Score

Hi-index	0.00

Visualization

Abstract

Ensemble methods are able to improve the predictive performance of many base classifiers. Up till now, they have been applied to classifiers that predict a single target attribute. Given the non-trivial interactions that may occur among the different targets in multi-objective prediction tasks, it is unclear whether ensemble methods also improve the performance in this setting. In this paper, we consider two ensemble learning techniques, bagging and random forests, and apply them to multi-objective decision trees (MODTs), which are decision trees that predict multiple target attributes at once. We empirically investigate the performance of ensembles of MODTs. Our most important conclusions are: (1) ensembles of MODTs yield better predictive performance than MODTs, and (2) ensembles of MODTs are equally good, or better than ensembles of single-objective decision trees, i.e., a set of ensembles for each target. Moreover, ensembles of MODTs have smaller model size and are faster to learn than ensembles of single-objective decision trees.