Choosing an evaluation metric for parser design

Authors:
Woodley Packard
Affiliations:
-
Venue:
NAACL HLT '12 Proceedings of the 2012 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies: Student Research Workshop
Year:
2012

Citing 4
Cited 0

Procedure for quantitatively comparing the syntactic coverage of English grammars

HLT '91 Proceedings of the workshop on Speech and Natural Language
A test of the leaf-ancestor metric for parse accuracy

Natural Language Engineering
On building a more efficient grammar by exploiting types

Natural Language Engineering
CoNLL-X shared task on multilingual dependency parsing

CoNLL-X '06 Proceedings of the Tenth Conference on Computational Natural Language Learning

Quantified Score

Hi-index	0.00

Visualization

Abstract

This paper seeks to quantitatively evaluate the degree to which a number of popular metrics provide overlapping information to parser designers. Two routine tasks are considered: optimizing a machine learning regularization parameter and selecting an optimal machine learning feature set. The main result is that the choice of evaluation metric used to optimize these problems (with one exception among popular metrics) has little effect on the solution to the optimization.