Training Conditional Random Fields by Periodic Step Size Adaptation for Large-Scale Text Mining

Authors:
Han-Shen Huang;Yu-Ming Chang;Chun-Nan Hsu
Affiliations:
-;-;-
Venue:
ICDM '07 Proceedings of the 2007 Seventh IEEE International Conference on Data Mining
Year:
2007

Citing 0
Cited 4

Global and componentwise extrapolations for accelerating training of Bayesian networks and conditional random fields

Data Mining and Knowledge Discovery
A simple feature-copying approach for long-distance dependencies

CoNLL '09 Proceedings of the Thirteenth Conference on Computational Natural Language Learning
Stochastic gradient descent training for L1-regularized log-linear models with cumulative penalty

ACL '09 Proceedings of the Joint Conference of the 47th Annual Meeting of the ACL and the 4th International Joint Conference on Natural Language Processing of the AFNLP: Volume 1 - Volume 1
Named entities in judicial transcriptions: extended conditional random fields

CICLing'13 Proceedings of the 14th international conference on Computational Linguistics and Intelligent Text Processing - Volume Part I

Quantified Score

Hi-index	0.00

Visualization

Abstract

For applications with consecutive incoming training examples, on-line learning has the potential to achieve a likelihood as high as off-line learning without scanning all available training examples and usually has a much smaller memory footprint. To train CRFs on-line, this paper presents the Periodic Step size Adaptation (PSA) method to dynamically adjust the learning rates in stochastic gradient descent. We applied our method to three large scale text mining tasks. Experimental results show that PSA outperforms the best off-line algorithm, L-BFGS, by many hundred times, and outperforms the best on-line algorithm, SMD, by an order of magnitude in terms of the number of passes required to scan the training data set.