The Hidden Information State model: A practical framework for POMDP-based spoken dialogue management

Authors:
Steve Young;Milica Gašić;Simon Keizer;François Mairesse;Jost Schatzmann;Blaise Thomson;Kai Yu
Affiliations:
Cambridge University Engineering Department, Trumpington Street, Cambridge, CB2 1PZ, UK;Cambridge University Engineering Department, Trumpington Street, Cambridge, CB2 1PZ, UK;Cambridge University Engineering Department, Trumpington Street, Cambridge, CB2 1PZ, UK;Cambridge University Engineering Department, Trumpington Street, Cambridge, CB2 1PZ, UK;Cambridge University Engineering Department, Trumpington Street, Cambridge, CB2 1PZ, UK;Cambridge University Engineering Department, Trumpington Street, Cambridge, CB2 1PZ, UK;Cambridge University Engineering Department, Trumpington Street, Cambridge, CB2 1PZ, UK
Venue:
Computer Speech and Language
Year:
2010

Citing 12
Cited 43

Planning and acting in partially observable stochastic domains

Artificial Intelligence
Planning and Acting under Uncertainty: A New Model for Spoken Dialogue System

UAI '01 Proceedings of the 17th Conference in Uncertainty in Artificial Intelligence
An epsilon-Optimal Grid-Based Algorithm for Partially Observable Markov Decision Processes

ICML '02 Proceedings of the Nineteenth International Conference on Machine Learning
Spoken dialogue management using probabilistic reasoning

ACL '00 Proceedings of the 38th Annual Meeting on Association for Computational Linguistics
Understanding spontaneous speech: the Phoenix system

ICASSP '91 Proceedings of the Acoustics, Speech, and Signal Processing, 1991. ICASSP-91., 1991 International Conference
Partially observable Markov decision processes for spoken dialog systems

Computer Speech and Language
Automating spoken dialogue management design using machine learning: An industry perspective

Speech Communication
Mixture model POMDPs for efficient handling of uncertainty in dialogue management

HLT-Short '08 Proceedings of the 46th Annual Meeting of the Association for Computational Linguistics on Human Language Technologies: Short Papers
Agenda-based user simulation for bootstrapping a POMDP dialogue system

NAACL-Short '07 Human Language Technologies 2007: The Conference of the North American Chapter of the Association for Computational Linguistics; Companion Volume, Short Papers
Training and evaluation of the HIS POMDP dialogue system in noise

SIGdial '08 Proceedings of the 9th SIGdial Workshop on Discourse and Dialogue
Point-based value iteration: an anytime algorithm for POMDPs

IJCAI'03 Proceedings of the 18th international joint conference on Artificial intelligence
Scaling POMDPs for Spoken Dialog Management

IEEE Transactions on Audio, Speech, and Language Processing

Estimating probability of correctness for ASR N-best lists

SIGDIAL '09 Proceedings of the SIGDIAL 2009 Conference: The 10th Annual Meeting of the Special Interest Group on Discourse and Dialogue
k-nearest neighbor Monte-Carlo control algorithm for POMDP-based dialogue systems

SIGDIAL '09 Proceedings of the SIGDIAL 2009 Conference: The 10th Annual Meeting of the Special Interest Group on Discourse and Dialogue
Using knowledge of misunderstandings to increase the robustness of spoken dialogue systems

Knowledge-Based Systems
Phrase-based statistical language generation using graphical models and active learning

ACL '10 Proceedings of the 48th Annual Meeting of the Association for Computational Linguistics
Towards relational POMDPs for adaptive dialogue management

ACLstudent '10 Proceedings of the ACL 2010 Student Research Workshop
Leveraging hidden dialogue state to select tutorial moves

IUNLPBEA '10 Proceedings of the NAACL HLT 2010 Fifth Workshop on Innovative Use of NLP for Building Educational Applications
Generation under uncertainty

INLG '10 Proceedings of the 6th International Natural Language Generation Conference
Prototype of Czech spoken dialog system with mixed initiative for railway information service

TSD'10 Proceedings of the 13th international conference on Text, speech and dialogue
Parameter estimation for agenda-based user simulation

SIGDIAL '10 Proceedings of the 11th Annual Meeting of the Special Interest Group on Discourse and Dialogue
Gaussian processes for fast policy optimisation of POMDP-based dialogue managers

SIGDIAL '10 Proceedings of the 11th Annual Meeting of the Special Interest Group on Discourse and Dialogue
Representing uncertainty about complex user goals in statistical dialogue systems

SIGDIAL '10 Proceedings of the 11th Annual Meeting of the Special Interest Group on Discourse and Dialogue
Investigating clarification strategies in a hybrid POMDP dialog manager

SIGDIAL '10 Proceedings of the 11th Annual Meeting of the Special Interest Group on Discourse and Dialogue
Dialogue act modeling in a complex task-oriented domain

SIGDIAL '10 Proceedings of the 11th Annual Meeting of the Special Interest Group on Discourse and Dialogue
Effective handling of dialogue state in the hidden information state POMDP-based dialogue manager

ACM Transactions on Speech and Language Processing (TSLP)
Spatially-aware dialogue control using hierarchical reinforcement learning

ACM Transactions on Speech and Language Processing (TSLP)
Natural actor and belief critic: Reinforcement algorithm for learning parameters of dialogue systems modelled as POMDPs

ACM Transactions on Speech and Language Processing (TSLP)
Learning and evaluation of dialogue strategies for new applications: Empirical methods for optimization from small data sets

Computational Linguistics
Reinforcement learning for parameter estimation in statistical spoken dialogue systems

Computer Speech and Language
The impact of task-oriented feature sets on HMMs for dialogue modeling

SIGDIAL '11 Proceedings of the SIGDIAL 2011 Conference
An empirical evaluation of a statistical dialog system in public use

SIGDIAL '11 Proceedings of the SIGDIAL 2011 Conference
Multi-policy dialogue management

SIGDIAL '11 Proceedings of the SIGDIAL 2011 Conference
Bootstrapping semantic parsers from conversations

EMNLP '11 Proceedings of the Conference on Empirical Methods in Natural Language Processing
Learning automata-based approach to learn dialogue policies in large state space

International Journal of Intelligent Information and Database Systems
Adaptive information presentation for spoken dialogue systems: evaluation with human subjects

ENLG '11 Proceedings of the 13th European Workshop on Natural Language Generation
Estimating a user’s internal state before the first input utterance

Advances in Human-Computer Interaction
An adaptive dialogue system with online dialogue policy learning

SETN'12 Proceedings of the 7th Hellenic conference on Artificial Intelligence: theories and applications
A statistical spoken dialogue system using complex user goals and value directed compression

EACL '12 Proceedings of the Demonstrations at the 13th Conference of the European Chapter of the Association for Computational Linguistics
A comparative study of reinforcement learning techniques on dialogue management

EACL '12 Proceedings of the Student Research Workshop at the 13th Conference of the European Chapter of the Association for Computational Linguistics
A belief tracking challenge task for spoken dialog systems

SDCTD '12 NAACL-HLT Workshop on Future Directions and Needs in the Spoken Dialog Community: Tools and Data
Optimising incremental dialogue decisions using information density for interactive systems

EMNLP-CoNLL '12 Proceedings of the 2012 Joint Conference on Empirical Methods in Natural Language Processing and Computational Natural Language Learning
The effect of cognitive load on a statistical dialogue system

SIGDIAL '12 Proceedings of the 13th Annual Meeting of the Special Interest Group on Discourse and Dialogue
Probabilistic dialogue models with prior domain knowledge

SIGDIAL '12 Proceedings of the 13th Annual Meeting of the Special Interest Group on Discourse and Dialogue
Exploiting machine-transcribed dialog corpus to improve multiple dialog states tracking methods

SIGDIAL '12 Proceedings of the 13th Annual Meeting of the Special Interest Group on Discourse and Dialogue
Integrating incremental speech recognition and POMDP-based dialogue systems

SIGDIAL '12 Proceedings of the 13th Annual Meeting of the Special Interest Group on Discourse and Dialogue
Tractable POMDP representations for intelligent tutoring systems

ACM Transactions on Intelligent Systems and Technology (TIST) - Special section on agent communication, trust in multiagent systems, intelligent tutoring and coaching systems
Using agents in virtual environments to assist controllers to manage multiple assets

CAVE'12 Proceedings of the First international conference on Cognitive Agents for Virtual Environments
Social signal and user adaptation in reinforcement learning-based dialogue management

Proceedings of the 2nd Workshop on Machine Learning for Interactive Systems: Bridging the Gap Between Perception, Action and Communication
A dialogue system for multimodal human-robot interaction

Proceedings of the 15th ACM on International conference on multimodal interaction
Spoken language processing: where do we go from here?

Your Virtual Butler
Towards action selection under uncertainty for a socially aware robot bartender

Proceedings of the 2014 ACM/IEEE international conference on Human-robot interaction
Attentional top-down regulation and dialogue management in human-robot interaction

Proceedings of the 2014 ACM/IEEE international conference on Human-robot interaction
Gaussian Processes for POMDP-Based Dialogue Manager Optimization

IEEE/ACM Transactions on Audio, Speech and Language Processing (TASLP)
A dialogue management system using a corpus-based framework and a dynamic dialogue transition model

AI Communications

Quantified Score

Hi-index	0.00

Visualization

Abstract

This paper explains how Partially Observable Markov Decision Processes (POMDPs) can provide a principled mathematical framework for modelling the inherent uncertainty in spoken dialogue systems. It briefly summarises the basic mathematics and explains why exact optimisation is intractable. It then describes in some detail a form of approximation called the Hidden Information State model which does scale and which can be used to build practical systems. A prototype HIS system for the tourist information domain is evaluated and compared with a baseline MDP system using both user simulations and a live user trial. The results give strong support to the central contention that the POMDP-based framework is both a tractable and powerful approach to building more robust spoken dialogue systems.