Towards standardized metrics and tools for spoken and multimodal dialog system evaluation: position paper

Authors:
Sebastian Möller;Klaus-Peter Engelbrecht;Florian Kretzschmar;Stefan Schmidt;Benjamin Weiss
Affiliations:
TU Berlin, Berlin, Germany;TU Berlin, Berlin, Germany;TU Berlin, Berlin, Germany;TU Berlin, Berlin, Germany;TU Berlin, Berlin, Germany
Venue:
SDCTD '12 NAACL-HLT Workshop on Future Directions and Needs in the Spoken Dialog Community: Tools and Data
Year:
2012

Citing 4
Cited 0

Towards a tool for the Subjective Assessment of Speech System Interfaces (SASSI)

Natural Language Engineering
PARADISE: a framework for evaluating spoken dialogue agents

ACL '98 Proceedings of the 35th Annual Meeting of the Association for Computational Linguistics and Eighth Conference of the European Chapter of the Association for Computational Linguistics
Predicting the quality and usability of spoken dialogue services

Speech Communication
Spoken Dialog Challenge 2010: comparison of live and control test results

SIGDIAL '11 Proceedings of the SIGDIAL 2011 Conference

Quantified Score

Hi-index	0.00

Visualization

Abstract

We argue that standardized metrics and automatic evaluation tools are necessary for speeding up knowledge generation and development processes for dialog systems.