Regularized Linear Models in Stacked Generalization

Authors:
Sam Reid;Greg Grudic
Affiliations:
University of Colorado at Boulder, Boulder, USA CO 80309-0430;University of Colorado at Boulder, Boulder, USA CO 80309-0430
Venue:
MCS '09 Proceedings of the 8th International Workshop on Multiple Classifier Systems
Year:
2009

Citing 8
Cited 4

Combination of Multiple Classifiers Using Local Accuracy Estimates

IEEE Transactions on Pattern Analysis and Machine Intelligence
Combining Pattern Classifiers: Methods and Algorithms

Combining Pattern Classifiers: Methods and Algorithms
The CoralReef Software Suite as a Tool for System and Network Administrators

LISA '01 Proceedings of the 15th USENIX conference on System administration
A preliminary performance comparison of five machine learning algorithms for practical IP traffic flow classification

ACM SIGCOMM Computer Communication Review
Early application identification

CoNEXT '06 Proceedings of the 2006 ACM CoNEXT conference
Pattern Recognition Approaches for Classifying IP Flows

SSPR & SPR '08 Proceedings of the 2008 Joint IAPR International Workshop on Structural, Syntactic, and Statistical Pattern Recognition
Traffic classification - towards accurate real time network applications

HCI'07 Proceedings of the 12th international conference on Human-computer interaction: applications and services
Bayesian Neural Networks for Internet Traffic Classification

IEEE Transactions on Neural Networks

Reranking for stacking ensemble learning

ICONIP'10 Proceedings of the 17th international conference on Neural information processing: theory and algorithms - Volume Part I
Empirical analysis and evaluation of approximate techniques for pruning regression bagging ensembles

Neurocomputing
Context-aware movie recommendation based on signal processing and machine learning

Proceedings of the 2nd Challenge on Context-Aware Movie Recommendation
Linear classifier combination and selection using group sparse regularization and hinge loss

Pattern Recognition Letters

Quantified Score

Hi-index	0.00

Visualization

Abstract

Stacked generalization is a flexible method for multiple classifier combination; however, it tends to overfit unless the combiner function is sufficiently smooth. Previous studies attempt to avoid overfitting by using a linear function at the combiner level. This paper demonstrates experimentally that even with a linear combination function, regularization is necessary to reduce overfitting and increase predictive accuracy. The standard linear least squares regression can be regularized with an L2 penalty (Ridge regression), an L1 penalty (lasso regression) or a combination of the two (elastic net regression). In multi-class classification, sparse linear models select and combine individual predicted probabilities instead of using complete probability distributions, allowing base classifiers to specialize in subproblems corresponding to different classes.