Confidence measures for dialogue management in the CU Communicator system

Authors:
R. San-Segundo;B. Pellom;W. Ward;J. M. Pardo
Affiliations:
Center for Spoken Language Understanding, Colorado Univ., Boulder, CO, USA;-;-;-
Venue:
ICASSP '00 Proceedings of the Acoustics, Speech, and Signal Processing, 2000. on IEEE International Conference - Volume 02
Year:
2000

Citing 0
Cited 5

University of Colorado dialog systems for travel and navigation

HLT '01 Proceedings of the first international conference on Human language technology research
Efficient dialogue strategy to find users' intended items from information query results

COLING '02 Proceedings of the 19th international conference on Computational linguistics - Volume 1
Efficient confirmation strategy for large-scale text retrieval systems with spoken dialogue interface

COLING '04 Proceedings of the 20th international conference on Computational Linguistics
Response-based confidence annotation for spoken dialogue systems

SIGdial '08 Proceedings of the 9th SIGdial Workshop on Discourse and Dialogue
Multi-domain spoken dialogue system with extensibility and robustness against speech recognition errors

SigDIAL '06 Proceedings of the 7th SIGdial Workshop on Discourse and Dialogue

Quantified Score

Hi-index	0.00

Visualization

Abstract

This paper provides improved confidence assessment for detection of word-level speech recognition errors and out-of-domain user requests using language model features. We consider a combined measure of confidence that utilizes the language model back-off sequence, language model score, and phonetic length of recognized words as indicators of speech recognition confidence. The paper investigates the ability of each feature to detect speech recognition errors and out-of-domain utterances as well as two methods for combining the features contextually: a multi-layer perceptron and a statistical decision tree. We illustrate the effectiveness of the algorithm by considering utterances from the ATIS airline information task as either in-domain and out-of-domain for the DARPA Communicator task. Using this hand-labeled data, it is shown that 27.9% of incorrectly recognized words and 36.4% of out-of-domain phrases are detected at a 2.5% false alarm rate.