Boosting with structural sparsity

Authors:
John Duchi;Yoram Singer
Affiliations:
University of California, Berkeley, CA;Google, Mountain View, CA
Venue:
ICML '09 Proceedings of the 26th Annual International Conference on Machine Learning
Year:
2009

Citing 8
Cited 11

Logistic Regression, AdaBoost and Bregman Distances

Machine Learning
An introduction to boosting and leveraging

Advanced lectures on machine learning
RCV1: A New Benchmark Collection for Text Categorization Research

The Journal of Machine Learning Research
Smooth ε-Insensitive Regression by Loss Symmetrization

The Journal of Machine Learning Research
Totally corrective boosting algorithms that maximize the margin

ICML '06 Proceedings of the 23rd international conference on Machine learning
On Model Selection Consistency of Lasso

The Journal of Machine Learning Research
Maximum Entropy Density Estimation with Generalized Regularization and an Application to Species Distribution Modeling

The Journal of Machine Learning Research
An Interior-Point Method for Large-Scale l1-Regularized Logistic Regression

The Journal of Machine Learning Research

Boosting with structure information in the functional space: an application to graph classification

Proceedings of the 16th ACM SIGKDD international conference on Knowledge discovery and data mining
Efficient and numerically stable sparse learning

ECML PKDD'10 Proceedings of the 2010 European conference on Machine learning and knowledge discovery in databases: Part III
A Comparison of Optimization Methods and Software for Large-scale L1-regularized Linear Classification

The Journal of Machine Learning Research
Content based social behavior prediction: a multi-task learning approach

Proceedings of the 20th ACM international conference on Information and knowledge management
Biomarker discovery using 1-norm regularization for multiclass earthworm microarray gene expression data

Neurocomputing
Weakly supervised learning of object segmentations from web-scale video

ECCV'12 Proceedings of the 12th international conference on Computer Vision - Volume Part I
Multi-Task boosting by exploiting task relationships

ECML PKDD'12 Proceedings of the 2012 European conference on Machine Learning and Knowledge Discovery in Databases - Volume Part I
Multi class learning with individual sparsity

IJCAI'13 Proceedings of the Twenty-Third international joint conference on Artificial Intelligence
Trust, but verify: predicting contribution quality for knowledge base construction and curation

Proceedings of the 7th ACM international conference on Web search and data mining
Block coordinate descent algorithms for large-scale sparse multiclass classification

Machine Learning
Fully corrective boosting with arbitrary loss and regularization

Neural Networks

Quantified Score

Hi-index	0.00

Visualization

Abstract

We derive generalizations of AdaBoost and related gradient-based coordinate descent methods that incorporate sparsity-promoting penalties for the norm of the predictor that is being learned. The end result is a family of coordinate descent algorithms that integrate forward feature induction and back-pruning through regularization and give an automatic stopping criterion for feature induction. We study penalties based on the l1, l2, and l∞ norms of the predictor and introduce mixed-norm penalties that build upon the initial penalties. The mixed-norm regularizers facilitate structural sparsity in parameter space, which is a useful property in multiclass prediction and other related tasks. We report empirical results that demonstrate the power of our approach in building accurate and structurally sparse models.