Better evaluation for grammatical error correction

Authors:
Daniel Dahlmeier;Hwee Tou Ng
Affiliations:
NUS Graduate School for Integrative Sciences and Engineering;NUS Graduate School for Integrative Sciences and Engineering and National University of Singapore
Venue:
NAACL HLT '12 Proceedings of the 2012 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies
Year:
2012

Citing 5
Cited 2

Introduction to algorithms

Introduction to algorithms
Information Retrieval

Information Retrieval
Building a large annotated corpus of English: the penn treebank

Computational Linguistics - Special issue on using large corpora: II
Automated Grammatical Error Detection for Language Learners

Automated Grammatical Error Detection for Language Learners
Helping our own: the HOO 2011 pilot shared task

ENLG '11 Proceedings of the 13th European Workshop on Natural Language Generation

HOO 2012: a report on the preposition and determiner error correction shared task

Proceedings of the Seventh Workshop on Building Educational Applications Using NLP
A beam-search decoder for grammatical error correction

EMNLP-CoNLL '12 Proceedings of the 2012 Joint Conference on Empirical Methods in Natural Language Processing and Computational Natural Language Learning

Quantified Score

Hi-index	0.00

Visualization

Abstract

We present a novel method for evaluating grammatical error correction. The core of our method, which we call MaxMatch (M2), is an algorithm for efficiently computing the sequence of phrase-level edits between a source sentence and a system hypothesis that achieves the highest overlap with the gold-standard annotation. This optimal edit sequence is subsequently scored using F1 measure. We test our M2 scorer on the Helping Our Own (HOO) shared task data and show that our method results in more accurate evaluation for grammatical error correction.