A three-tiered evaluation approach for interactive spoken dialogue systems

Authors:
Kathleen Stibler;James Denny
Affiliations:
Lockheed Martin Advanced Technology Laboratories, Camden NJ;Lockheed Martin Advanced Technology Laboratories, Camden NJ
Venue:
HLT '01 Proceedings of the first international conference on Human language technology research
Year:
2001

Citing 2
Cited 2

Overview of evaluation in speech and natural language processing

Survey of the state of the art in human language technology
PARADISE: a framework for evaluating spoken dialogue agents

ACL '98 Proceedings of the 35th Annual Meeting of the Association for Computational Linguistics and Eighth Conference of the European Chapter of the Association for Computational Linguistics

Listen-Communicate-Show (LCS): spoken language command of agent-based remote information access

HLT '01 Proceedings of the first international conference on Human language technology research
Towards a method for evaluating naturalness in conversational dialog systems

SMC'09 Proceedings of the 2009 IEEE international conference on Systems, Man and Cybernetics

Quantified Score

Hi-index	0.00

Visualization

Abstract

We describe a three-tiered approach for evaluation of spoken dialogue systems. The three tiers measure user satisfaction, system support of mission success and component performance. We describe our use of this approach in numerous fielded user studies conducted with the U.S. military.