Evaluation of a hierarchical reinforcement learning spoken dialogue system
Computer Speech and Language
| Hi-index | 0.01 |
A spoken dialogue system (SDS) is an application supporting conversational interaction with a human to perform some task. One barrier to the widespread deployment of SDSs exists in the form of communication breakdown in the dialogue, often when the user tries to supply a named entity from a large or open vocabulary set. For example, a weather information system will likely misrecognize an unknown city as some known city, leading to unpredictable system and user behavior. This thesis presents a framework for developing attribute acquisition strategies with a simulated user. We focus on acquiring unknown city names in a flight domain, through a spell-mode subdialogue. Collecting data from real users is costly. Furthermore, our goal is to focus on user behavior that tends to arise sporadically in real dialogues. Therefore, we employed user simulation to efficiently produce many dialogues, and to configure the input as desired in order to exercise specific strategies. We present a novel method of utterance generation for user simulation, combining segments of real and synthesized speech. This method retains the structural variety of real user utterances, while introducing synthesized items, such as unknown city names, that lead to problematic situations, for which we can develop strategies. We also devised a model of generic dialogue management, allowing a developer to quickly specify interaction properties on a per-attribute basis. This model was used to assess the effectiveness of various combinations of dialogue strategies and simulated user behavior. We use simulation to address two problems: the conflict problem requires the system to choose how to act when a new hypothesis for an attribute conflicts with its current belief, while the compliance problem requires the system to decide whether a user complied with a spelling request. Decision models were learned from simulated data, and were tested with real users, showing that the learned model significantly outperformed a heuristic model in choosing the "ideal" response to the conflict problem. The compliance prediction model achieved near perfect accuracy. These results suggest that such models learned from simulated data can attain similar, if not better, performance in dialogues with real users. (Copies available exclusively from MIT Libraries, Rm. 14-0551, Cambridge, MA 02139-4307. Ph. 617-253-5668; Fax 617-253-1690.)