The ATIS spoken language systems pilot corpus
HLT '90 Proceedings of the workshop on Speech and Natural Language
Developing an evaluation methodology for spoken language systems
HLT '90 Proceedings of the workshop on Speech and Natural Language
Beyond class A: a proposal for automatic evaluation of discourse
HLT '90 Proceedings of the workshop on Speech and Natural Language
A practical methodology for the evaluation of spoken language systems
ANLC '92 Proceedings of the third conference on Applied natural language processing
Progress report on the Chronus system: ATIS benchmark results
HLT '91 Proceedings of the workshop on Speech and Natural Language
HLT '90 Proceedings of the workshop on Speech and Natural Language
A speech understanding system based on statistical representation of semantics
ICASSP'92 Proceedings of the 1992 IEEE international conference on Acoustics, speech and signal processing - Volume 1
The estimation of powerful language models from small and large corpora
ICASSP'93 Proceedings of the 1993 IEEE international conference on Acoustics, speech, and signal processing: speech processing - Volume II
| Hi-index | 0.00 |
This paper proposes an automatic, essentially domain-independent means of evaluating Spoken Language Systems (SLS) which combines software we have developed for that purpose (the "Comparator") and a sert of specifications for answer expressions (the "Common Answer Specification", or CAS). The Comparator checks whether the answer provided by a SLS accords with a canonical answer, returning either true or false. The Common Answer Specification determines the syntax of answer expressions, the minimal content that must be included in them, the data to be included in and excluded from test corpora, and the procedures used by the Comparator. Though some details of the CAS are particular to individual domains, the Comparator software is domain-independent, as is the CAS approach.