BLEU: a method for automatic evaluation of machine translation
ACL '02 Proceedings of the 40th Annual Meeting on Association for Computational Linguistics
The Proposition Bank: An Annotated Corpus of Semantic Roles
Computational Linguistics
Robust PCFG-based generation using automatically acquired LFG approximations
ACL-44 Proceedings of the 21st International Conference on Computational Linguistics and the 44th annual meeting of the Association for Computational Linguistics
The CoNLL-2008 shared task on joint parsing of syntactic and semantic dependencies
CoNLL '08 Proceedings of the Twelfth Conference on Computational Natural Language Learning
Evaluating coverage for large symbolic NLG grammars
IJCAI'03 Proceedings of the 18th international joint conference on Artificial intelligence
Probabilistic models for disambiguation of an HPSG-based chart generator
Parsing '05 Proceedings of the Ninth International Workshop on Parsing Technology
Perceptron reranking for CCG realization
EMNLP '09 Proceedings of the 2009 Conference on Empirical Methods in Natural Language Processing: Volume 1 - Volume 1
Finding common ground: towards a surface realisation shared task
INLG '10 Proceedings of the 6th International Natural Language Generation Conference
Broad coverage multilingual deep sentence generation with a stochastic multi-level realizer
COLING '10 Proceedings of the 23rd International Conference on Computational Linguistics
Discrete vs. continuous rating scales for language evaluation in NLP
HLT '11 Proceedings of the 49th Annual Meeting of the Association for Computational Linguistics: Human Language Technologies: short papers - Volume 2
Meteor 1.3: automatic metric for reliable optimization and evaluation of machine translation systems
WMT '11 Proceedings of the Sixth Workshop on Statistical Machine Translation
Error mining on dependency trees
ACL '12 Proceedings of the 50th Annual Meeting of the Association for Computational Linguistics: Long Papers - Volume 1
Minimal dependency length in realization ranking
EMNLP-CoNLL '12 Proceedings of the 2012 Joint Conference on Empirical Methods in Natural Language Processing and Computational Natural Language Learning
Generating non-projective word order in statistical linearization
EMNLP-CoNLL '12 Proceedings of the 2012 Joint Conference on Empirical Methods in Natural Language Processing and Computational Natural Language Learning
Towards a surface realization-oriented corpus annotation
INLG '12 Proceedings of the Seventh International Natural Language Generation Conference
Generation for grammar engineering
INLG '12 Proceedings of the Seventh International Natural Language Generation Conference
The surface realisation task: recent developments and future plans
INLG '12 Proceedings of the Seventh International Natural Language Generation Conference
Partial-tree linearization: generalized word ordering for text synthesis
IJCAI'13 Proceedings of the Twenty-Third international joint conference on Artificial Intelligence
| Hi-index | 0.00 |
The Surface Realisation (SR) Task was a new task at Generation Challenges 2011, and had two tracks: (1) Shallow: mapping from shallow input representations to realisations; and (2) Deep: mapping from deep input representations to realisations. Five teams submitted six systems in total, and we additionally evaluated human toplines. Systems were evaluated automatically using a range of intrinsic metrics. In addition, systems were assessed by human judges in terms of Clarity, Readability and Meaning Similarity. This report presents the evaluation results, along with descriptions of the SR Task Tracks and evaluation methods. For descriptions of the participating systems, see the separate system reports in this volume, immediately following this results report.