Bias and variance of rotation-based ensembles

Authors:
Juan José Rodríguez;Carlos J. Alonso;Oscar J. Prieto
Affiliations:
Lenguajes y Sistemas Informáticos, Universidad de Burgos, Spain;Grupo de Sistemas Inteligentes, Departamento de Informática, Universidad de Valladolid, Spain;Grupo de Sistemas Inteligentes, Departamento de Informática, Universidad de Valladolid, Spain
Venue:
IWANN'05 Proceedings of the 8th international conference on Artificial Neural Networks: computational Intelligence and Bioinspired Systems
Year:
2005

Citing 11
Cited 3

C4.5: programs for machine learning

C4.5: programs for machine learning
Bagging predictors

Machine Learning
The Random Subspace Method for Constructing Decision Forests

IEEE Transactions on Pattern Analysis and Machine Intelligence
Data mining: practical machine learning tools and techniques with Java implementations

Data mining: practical machine learning tools and techniques with Java implementations
Data mining: concepts and techniques

Data mining: concepts and techniques
Random Forests

Machine Learning
An Empirical Comparison of Voting Classification Algorithms: Bagging, Boosting, and Variants

Machine Learning
A Unifeid Bias-Variance Decomposition and its Applications

ICML '00 Proceedings of the Seventeenth International Conference on Machine Learning
Ensemble Methods in Machine Learning

MCS '00 Proceedings of the First International Workshop on Multiple Classifier Systems
Combining Pattern Classifiers: Methods and Algorithms

Combining Pattern Classifiers: Methods and Algorithms
Not So Naive Bayes: Aggregating One-Dependence Estimators

Machine Learning

RotBoost: A technique for combining Rotation Forest and AdaBoost

Pattern Recognition Letters
Stop wasting time: on predicting the success or failure of learning for industrial applications

IDEAL'07 Proceedings of the 8th international conference on Intelligent data engineering and automated learning
DF-SVM: a decision forest constructed on artificially enlarged feature space by support vector machine

Artificial Intelligence Review

Quantified Score

Hi-index	0.00

Visualization

Abstract

In Machine Learning, ensembles are combination of classifiers. Their objective is to improve the accuracy. In previous works, we have presented a method for the generation of ensembles, named rotation-based. It transforms the training data set; it groups, randomly, the attributes in different subgroups, and applies, for each group, an axis rotation. If the used method for the induction of the classifiers is not invariant to rotations in the data set, the generated classifiers can be very different. In this way, different classifiers can be obtained (and combined) using the same induction method. The bias-variance decomposition of the error is used to get some insight into the behaviour of a classifier. It has been used to explain the success of ensemble learning techniques. In this work the bias and variance for the presented and other ensemble methods are calculated and used for comparison purposes.