Iterative scaling and coordinate descent methods for maximum entropy

Authors:
Fang-Lan Huang;Cho-Jui Hsieh;Kai-Wei Chang;Chih-Jen Lin
Affiliations:
National Taiwan University, Taipai, Taiwan;National Taiwan University, Taipai, Taiwan;National Taiwan University, Taipai, Taiwan;National Taiwan University, Taipai, Taiwan
Venue:
ACLShort '09 Proceedings of the ACL-IJCNLP 2009 Conference Short Papers
Year:
2009

Citing 4
Cited 1

Inducing Features of Random Fields

IEEE Transactions on Pattern Analysis and Machine Intelligence
Sequential conditional Generalized Iterative Scaling

ACL '02 Proceedings of the 40th Annual Meeting on Association for Computational Linguistics
A comparison of algorithms for maximum entropy parameter estimation

COLING-02 proceedings of the 6th conference on Natural language learning - Volume 20
Coordinate Descent Method for Large-scale L2-loss Linear Support Vector Machines

The Journal of Machine Learning Research

Iterative Scaling and Coordinate Descent Methods for Maximum Entropy Models

The Journal of Machine Learning Research

Quantified Score

Hi-index	0.00

Visualization

Abstract

Maximum entropy (Maxent) is useful in many areas. Iterative scaling (IS) methods are one of the most popular approaches to solve Maxent. With many variants of IS methods, it is difficult to understand them and see the differences. In this paper, we create a general and unified framework for IS methods. This framework also connects IS and coordinate descent (CD) methods. Besides, we develop a CD method for Maxent. Results show that it is faster than existing iterative scaling methods.