On building a more efficient grammar by exploiting types
Natural Language Engineering
A compact architecture for dialogue management based on scripts and meta-outputs
ANLC '00 Proceedings of the sixth conference on Applied natural language processing
A maximum-entropy-inspired parser
NAACL 2000 Proceedings of the 1st North American chapter of the Association for Computational Linguistics conference
Processing unknown words in HPSG
COLING '98 Proceedings of the 17th international conference on Computational linguistics - Volume 1
Lexicon acquisition with a large-coverage unification-based grammar
EACL '03 Proceedings of the tenth conference on European chapter of the Association for Computational Linguistics - Volume 2
The LinGO Redwoods treebank motivation and preliminary applications
COLING '02 Proceedings of the 19th international conference on Computational linguistics - Volume 2
Efficient deep processing of Japanese
COLING '02 Proceedings of the 3rd workshop on Asian language resources and international standardization - Volume 12
Error mining for wide-coverage grammar engineering
ACL '04 Proceedings of the 42nd Annual Meeting on Association for Computational Linguistics
Creating a CCGbank and a wide-coverage CCG lexicon for German
ACL-44 Proceedings of the 21st International Conference on Computational Linguistics and the 44th annual meeting of the Association for Computational Linguistics
Bootstrapping deep lexical resources: resources for courses
DeepLA '05 Proceedings of the ACL-SIGLEX Workshop on Deep Lexical Acquisition
Automatically extracting and comparing lexicalized grammars for different languages
IJCAI'01 Proceedings of the 17th international joint conference on Artificial intelligence - Volume 2
The hinoki treebank a treebank for text understanding
IJCNLP'04 Proceedings of the First international joint conference on Natural Language Processing
A flexible approach to class-based ordering of prenominal modifiers
Empirical methods in natural language generation
| Hi-index | 0.00 |
This paper is concerned with the standardisation of evaluation metrics for lexical acquisition over precision grammars, which are attuned to actual parser performance. Specifically, we investigate the impact that lexicons at varying levels of lexical item precision and recall have on the performance of pre-existing broad-coverage precision grammars in parsing, i.e., on their coverage and accuracy. The grammars used for the experiments reported here are the LinGO English Resource Grammar (ERG; Flickinger (2000)) and JaCY (Siegel and Bender, 2002), precision grammars of English and Japanese, respectively. Our results show convincingly that traditional F-score-based evaluation of lexical acquisition does not correlate with actual parsing performance. What we argue for, therefore, is a recall-heavy interpretation of F-score in designing and optimising automated lexical acquisition algorithms.