On selection and combination of weak learners in AdaBoost
Pattern Recognition Letters
| Hi-index | 0.00 |
Many classification algorithms (such as Adaboost) aim to minimize training error, that is, the ratio of the number of mis-classified examples to the total number of examples. However, it is often desirable to minimize a more general cost metric, where distinct examples have different costs. For example, in industrial inspection, the cost of clearing a defective part is substantially greater than the cost of incorrectly alarming on a good part. Further, defects themselves vary in severity and, hence, cost. Extending boosting algorithms to account for this variable cost (cost-sensitive boosting) has been attempted by a number of investigators, resulting in a number of Adaboost variants, including AdaCost, Asymboost, CSB0, CSB1, and CSB2. These algorithms all focus only on the distribution update step in Adaboost-leaving the error computation step unchanged (or only indirectly modified). In this paper, we present a novel algorithm, termed "distributed cost boosting" (DCB), that incorporates the cost metric into the error computation step, rather than merely adjusting the distribution update step. DCB produces large empirical reductions in total cost when compared to current state-ofthe- art cost-sensitive Adaboost variants. Further, we show that distributed cost boosting provides error bounds on misclassification cost similar to AdaBoost's bounds on training error rate.