Selective Ensemble under Regularization Framework

Authors:
Nan Li;Zhi-Hua Zhou
Affiliations:
National Key Laboratory for Novel Software Technology, Nanjing University, Nanjing, China 210093;National Key Laboratory for Novel Software Technology, Nanjing University, Nanjing, China 210093
Venue:
MCS '09 Proceedings of the 8th International Workshop on Multiple Classifier Systems
Year:
2009

Citing 10
Cited 4

Probabilistic neural networks

Neural Networks
Original Contribution: Stacked generalization

Neural Networks
Stacked regressions

Machine Learning
Random Forests

Machine Learning
Neural Network Methodology for 1H NMR Spectroscopy Classification

ICIIS '99 Proceedings of the 1999 International Conference on Information Intelligence and Systems
A note on Platt's probabilistic outputs for support vector machines

Machine Learning
Overfitting in the selection of classifier ensembles: a comparative study between PSO and GA

Proceedings of the 10th annual conference on Genetic and evolutionary computation
Automatic Classification of NMR Spectra by Ensembles of Local Experts

SSPR & SPR '08 Proceedings of the 2008 Joint IAPR International Workshop on Structural, Syntactic, and Statistical Pattern Recognition
Issues in stacked generalization

Journal of Artificial Intelligence Research
On the application of SVM-ensembles based on adapted random subspace sampling for automatic classification of NMR data

MCS'07 Proceedings of the 7th international conference on Multiple classifier systems

Margin optimization based pruning for random forest

Neurocomputing
Selective ensemble of support vector data descriptions for novelty detection

ISNN'12 Proceedings of the 9th international conference on Advances in Neural Networks - Volume Part I
Diversity regularized ensemble pruning

ECML PKDD'12 Proceedings of the 2012 European conference on Machine Learning and Knowledge Discovery in Databases - Volume Part I
A novel classifier ensemble method with sparsity and diversity

Neurocomputing

Quantified Score

Hi-index	0.00

Visualization

Abstract

An ensemble is generated by training multiple component learners for a same task and then combining them for predictions. It is known that when lots of trained learners are available, it is better to ensemble some instead of all of them. The selection, however, is generally difficult and heuristics are often used. In this paper, we investigate the problem under the regularization framework, and propose a regularized selective ensemble algorithm RSE. In RSE, the selection is reduced to a quadratic programming problem, which has a sparse solution and can be solved efficiently. Since it naturally fits the semi-supervised learning setting, RSE can also exploit unlabeled data to improve the performance. Experimental results show that RSE can generate ensembles with small size but strong generalization ability.