Dual coordinate descent methods for logistic regression and maximum entropy models

Authors:
Hsiang-Fu Yu;Fang-Lan Huang;Chih-Jen Lin
Affiliations:
Department of Computer Science, National Taiwan University, Taipei, Taiwan 106;Department of Computer Science, National Taiwan University, Taipei, Taiwan 106;Department of Computer Science, National Taiwan University, Taipei, Taiwan 106
Venue:
Machine Learning
Year:
2011

Citing 0
Cited 2

Selective block minimization for faster convergence of limited memory large-scale linear models

Proceedings of the 17th ACM SIGKDD international conference on Knowledge discovery and data mining
Exploring discriminatory features for automated malware classification

DIMVA'13 Proceedings of the 10th international conference on Detection of Intrusions and Malware, and Vulnerability Assessment

Quantified Score

Hi-index	0.00

Visualization

Abstract

Most optimization methods for logistic regression or maximum entropy solve the primal problem. They range from iterative scaling, coordinate descent, quasi-Newton, and truncated Newton. Less efforts have been made to solve the dual problem. In contrast, for linear support vector machines (SVM), methods have been shown to be very effective for solving the dual problem. In this paper, we apply coordinate descent methods to solve the dual form of logistic regression and maximum entropy. Interestingly, many details are different from the situation in linear SVM. We carefully study the theoretical convergence as well as numerical issues. The proposed method is shown to be faster than most state of the art methods for training logistic regression and maximum entropy.