Logistic Regression, AdaBoost and Bregman Distances
Machine Learning
| Hi-index | 0.00 |
In 2001, Della Pietra, Della Pietra, and Lafferty suggested a dual characterization of the Bregman projection onto linear constraints, which has already been applied by Collins, Schapire, and Singer to boosting algorithms and maximum likelihood logistic regression. The proof provided by Della Pietra et al. is fairly complicated, and their statement features a curious nonconvex component.In this note, the Della Pietra et al. characterization is proved differently, using the powerful framework of convex analysis. Assuming a standard constraint qualification, the proof presented here is not only much shorter and cleaner, but it also reveals the strange nonconvex component as a reformulation of a convex (dual) optimization problem. Furthermore, the setting is extended from an affine subspace to a translated cone, and the convex function inducing the Bregman distance is only required to be Legendre. Various remarks are made on limitations and possible extensions.