Boosting strategy for classification

Authors:
Huma Lodhi;Grigoris Karakoulas;John Shawe-Taylor
Affiliations:
Department of Computer Science, Royal Holloway, University of London, Egham, Surrey TW20 0EX, UK. E-mail: huma@cs.rhul.ac.uk;Global Analytics Group, Canadian Imperial Bank of Commerce, 161 Bay St., BCE-11, Toronto ON, Canada M5J 2S8. E-mail: Grigoris.Karakoulas@cibc.ca;Department of Computer Science, Royal Holloway, University of London, Egham, Surrey TW20 0EX, UK. E-mail: john@cs.rhul.ac.uk
Venue:
Intelligent Data Analysis
Year:
2002

Citing 15
Cited 2

A training algorithm for optimal margin classifiers

COLT '92 Proceedings of the fifth annual workshop on Computational learning theory
Support-Vector Networks

Machine Learning
Inductive learning algorithms and representations for text categorization

Proceedings of the seventh international conference on Information and knowledge management
Boosting and Rocchio applied to text filtering

Proceedings of the 21st annual international ACM SIGIR conference on Research and development in information retrieval
Boosting in the limit: maximizing the margin of learned ensembles

AAAI '98/IAAI '98 Proceedings of the fifteenth national/tenth conference on Artificial intelligence/Innovative applications of artificial intelligence
Training methods for adaptive boosting of neural networks

NIPS '97 Proceedings of the 1997 conference on Advances in neural information processing systems 10
Further results on the margin distribution

COLT '99 Proceedings of the twelfth annual conference on Computational learning theory
BoosTexter: A Boosting-based Systemfor Text Categorization

Machine Learning - Special issue on information retrieval
An Experimental Comparison of Three Methods for Constructing Ensembles of Decision Trees: Bagging, Boosting, and Randomization

Machine Learning
Soft Margins for AdaBoost

Machine Learning
An Empirical Comparison of Voting Classification Algorithms: Bagging, Boosting, and Variants

Machine Learning
Text Categorization with Suport Vector Machines: Learning with Many Relevant Features

ECML '98 Proceedings of the 10th European Conference on Machine Learning
A Column Generation Algorithm For Boosting

ICML '00 Proceedings of the Seventeenth International Conference on Machine Learning
An empirical evaluation of bagging and boosting

AAAI'97/IAAI'97 Proceedings of the fourteenth national conference on artificial intelligence and ninth conference on Innovative applications of artificial intelligence
Bagging, boosting, and C4.S

AAAI'96 Proceedings of the thirteenth national conference on Artificial intelligence - Volume 1

Computing confidence measures in stochastic logic programs

MICAI'05 Proceedings of the 4th Mexican international conference on Advances in Artificial Intelligence
Modelling metabolic pathways using stochastic logic programs-based ensemble methods

CMSB'04 Proceedings of the 20 international conference on Computational Methods in Systems Biology

Quantified Score

Hi-index	0.00

Visualization

Abstract

This paper introduces a strategy for training ensemble classifiers by analysing boosting within margin theory. We present a bound on the generalisation error of ensembled classifiers in terms of the 2-norm of the margin slack vector. We develop an effective, adaptive and robust boosting algorithm, DMBoost, by optimising this bound. The soft margin based quadratic loss function is insensitive to points having a large margin. The algorithm improves the generalisation performance of a system by ignoring the examples having small or negative margin. We evaluate the efficacy of the proposed method by applying it to a text categorization task. Experimental results show that DMBoost performs significantly better than AdaBoost, hence validating the effectiveness of the method. Furthermore, experimental results on UCI data sets demonstrate that DMBoost generally outperforms AdaBoost.