The need for accurate alignment in natural language system evaluation
Computational Linguistics
Task tolerance of MT output in integrated text processes
NAACL-ANLP-EMTS '00 Proceedings of the 2000 NAACL-ANLP Workshop on Embedded machine translation systems - Volume 5
Automatic extraction of morphological information from botanical collections
Proceedings of the 8th ACM/IEEE-CS joint conference on Digital libraries
A survey on session detection methods in query logs and a proposal for future evaluation
Information Sciences: an International Journal
Task tolerance of MT output in integrated text processes
EmbedMT '00 ANLP-NAACL 2000 Workshop: Embedded Machine Translation Systems
Retrieval effectiveness of cross language information retrieval search engines
ICADL'11 Proceedings of the 13th international conference on Asia-pacific digital libraries: for cultural heritage, knowledge dissemination, and future creation
| Hi-index | 0.00 |
The MUC-6 scoring method is based on a two-step process of mapping an item generated by a system under evaluation (the "response") to the corresponding item in the human-generated answer key and then scoring the mapped items. The resulting scores are used for decision-making over the entire evaluation cycle, including refinement of the task definition based on interannotator comparisons, technology development using training data, validating answer keys, and benchmarking both system and human capabilities on the test data.