Increasing the Robustness of Boosting Algorithms within the Linear-programming Framework

Authors:
Yijun Sun;Sinisa Todorovic;Jian Li
Affiliations:
Interdisciplinary Center for Biotechnology Research, University of Florida, Gainesville, USA 32610-3622;Beckman Institute for Advanced Science and Technology, University of Illinois at Urbana-Champaign, Urbana, USA 61801;Department of Electrical and Computer Engineering, University of Florida, Gainesville, USA 32610
Venue:
Journal of VLSI Signal Processing Systems
Year:
2007

Citing 20
Cited 0

C4.5: programs for machine learning

C4.5: programs for machine learning
Support-Vector Networks

Machine Learning
Bagging predictors

Machine Learning
Game theory, on-line prediction and boosting

COLT '96 Proceedings of the ninth annual conference on Computational learning theory
A decision-theoretic generalization of on-line learning and an application to boosting

Journal of Computer and System Sciences - Special issue: 26th annual ACM symposium on the theory of computing & STOC'94, May 23–25, 1994, and second annual Europe an conference on computational learning theory (EuroCOLT'95), March 13–15, 1995
Boosting in the limit: maximizing the margin of learned ensembles

AAAI '98/IAAI '98 Proceedings of the fifteenth national/tenth conference on Artificial intelligence/Innovative applications of artificial intelligence
Prediction games and arcing algorithms

Neural Computation
Improved Boosting Algorithms Using Confidence-rated Predictions

Machine Learning - The Eleventh Annual Conference on computational Learning Theory
An Experimental Comparison of Three Methods for Constructing Ensembles of Decision Trees: Bagging, Boosting, and Randomization

Machine Learning
Soft Margins for AdaBoost

Machine Learning
Neural Networks for Pattern Recognition

Neural Networks for Pattern Recognition
Linear Programming Boosting via Column Generation

Machine Learning
Some Theoretical Aspects of Boosting in the Presence of Noisy Data

ICML '01 Proceedings of the Eighteenth International Conference on Machine Learning
Boosting the margin: A new explanation for the effectiveness of voting methods

ICML '97 Proceedings of the Fourteenth International Conference on Machine Learning
Using output codes to boost multiclass learning problems

ICML '97 Proceedings of the Fourteenth International Conference on Machine Learning
An introduction to boosting and leveraging

Advanced lectures on machine learning
The Dynamics of AdaBoost: Cyclic Behavior and Convergence of Margins

The Journal of Machine Learning Research
Aggregate features and ADABOOST for music classification

Machine Learning
Using boosting to improve a hybrid HMM/neural network speech recognizer

ICASSP '99 Proceedings of the Acoustics, Speech, and Signal Processing, 1999. on 1999 IEEE International Conference - Volume 02
Automatic hardware implementation tool for a discrete Adaboost-based decision algorithm

EURASIP Journal on Applied Signal Processing

Quantified Score

Hi-index	0.00

Visualization

Abstract

AdaBoost has been successfully used in many signal classification systems. However, it has been observed that on highly noisy data AdaBoost easily leads to overfitting, which seriously constrains its applicability. In this paper, we address this problem by proposing a new regularized boosting algorithm LPnorm2-AdaBoost (LPNA). This algorithm arises from a close connection between AdaBoost and linear programming. In the algorithm, skewness of the data distribution is controlled during the training to prevent outliers from spoiling decision boundaries. To this end, a smooth convex penalty function (l 2 norm) is introduced in the objective function of a minimax problem. A stabilized column generation technique is used to transform the optimization problem into a simple linear programming problem. The effectiveness of the proposed algorithm is demonstrated through experiments on many diverse datasets.