Procedure for quantitatively comparing the syntactic coverage of English grammars
HLT '91 Proceedings of the workshop on Speech and Natural Language
Natural language parsing as statistical pattern recognition
Natural language parsing as statistical pattern recognition
Tree-bank Grammars
Building a large annotated corpus of English: the penn treebank
Computational Linguistics - Special issue on using large corpora: II
An automatic treebank conversion algorithm for corpus sharing
ACL '94 Proceedings of the 32nd annual meeting on Association for Computational Linguistics
Converting dependency structures to phrase structures
HLT '01 Proceedings of the first international conference on Human language technology research
A common framework for syntactic annotation
ACL '01 Proceedings of the 39th Annual Meeting on Association for Computational Linguistics
Timber! issues in treebank building and use
PROPOR'03 Proceedings of the 6th international conference on Computational processing of the Portuguese language
| Hi-index | 0.00 |
Despite its great importance to developing parsing systems, the task of evaluating the performance of a syntactic parser of natural language is poorly defined. This paper provides a survey of parser evaluation methods and outlines a framework for experimental parser evaluation. Clearly, there is a lack of a comprehensive evaluation framework and a generic evaluation tool for parsers in the research community. Several evaluation methods exist and some practical evaluations have been carried out, but they usually concentrate on a single level of parsers’ performance. The proposed framework focuses on intrinsic evaluation, providing useful information for parser developers. We provide a fuller picture of parser’s performance compared to using the standard precision and recall measures. In addition, we consider ways of using the framework for comparative evaluations. The main motivation for this work is to serve as a requirements analysis for a parser evaluation tool to be implemented.