Automatic labeling of semantic roles
Computational Linguistics
Using predicate-argument structures for information extraction
ACL '03 Proceedings of the 41st Annual Meeting on Association for Computational Linguistics - Volume 1
The Penn Treebank: annotating predicate argument structure
HLT '94 Proceedings of the workshop on Human Language Technology
The Proposition Bank: An Annotated Corpus of Semantic Roles
Computational Linguistics
A global joint model for semantic role labeling
Computational Linguistics
Tree kernels for semantic role labeling
Computational Linguistics
Semantic role assignment for event nominalisations by leveraging verbal data
COLING '08 Proceedings of the 22nd International Conference on Computational Linguistics - Volume 1
Semi-supervised semantic role labeling
EACL '09 Proceedings of the 12th Conference of the European Chapter of the Association for Computational Linguistics
Automatic induction of FrameNet lexical units
EMNLP '08 Proceedings of the Conference on Empirical Methods in Natural Language Processing
SemEval'07 task 19: frame semantic structure extraction
SemEval '07 Proceedings of the 4th International Workshop on Semantic Evaluations
SemEval-2010 task 10: linking events and their participants in discourse
DEW '09 Proceedings of the Workshop on Semantic Evaluations: Recent Achievements and Future Directions
Combining word sense and usage for modeling frame semantics
STEP '08 Proceedings of the 2008 Conference on Semantics in Text Processing
Assessing the impact of frame semantics on textual entailment
Natural Language Engineering
Graph alignment for semi-supervised semantic role labeling
EMNLP '09 Proceedings of the 2009 Conference on Empirical Methods in Natural Language Processing: Volume 1 - Volume 1
Semi-supervised semantic role labeling using the latent words language model
EMNLP '09 Proceedings of the 2009 Conference on Empirical Methods in Natural Language Processing: Volume 1 - Volume 1
A Bayesian model for unsupervised semantic parsing
HLT '11 Proceedings of the 49th Annual Meeting of the Association for Computational Linguistics: Human Language Technologies - Volume 1
A Bayesian approach to unsupervised semantic role induction
EACL '12 Proceedings of the 13th Conference of the European Chapter of the Association for Computational Linguistics
Unsupervised induction of frame-semantic representations
WILS '12 Proceedings of the NAACL-HLT Workshop on the Induction of Linguistic Structure
| Hi-index | 0.00 |
Supervised semantic role labeling (SRL) systems are generally claimed to have accuracies in the range of 80% and higher (Erk and Padó, 2006). These numbers, though, are the result of highly-restricted evaluations, i.e., typically evaluating on hand-picked lemmas for which training data is available. In this paper we consider performance of such systems when we evaluate at the document level rather than on the lemma level. While it is well-known that coverage gaps exist in the resources available for training supervised SRL systems, what we have been lacking until now is an understanding of the precise nature of this coverage problem and its impact on the performance of SRL systems. We present a typology of five different types of coverage gaps in FrameNet. We then analyze the impact of the coverage gaps on performance of a supervised semantic role labeling system on full texts, showing an average oracle upper bound of 46.8%.