C4.5: programs for machine learning
C4.5: programs for machine learning
Empirically evaluating an adaptable spoken dialogue system
UM '99 Proceedings of the seventh international conference on User modeling
Reinforcement Learning
Speech and Language Processing: An Introduction to Natural Language Processing, Computational Linguistics, and Speech Recognition
Separating Skills from Preference: Using Learning to Program by Reward
ICML '02 Proceedings of the Nineteenth International Conference on Machine Learning
Correlation-based Feature Selection for Discrete and Numeric Class Machine Learning
ICML '00 Proceedings of the Seventeenth International Conference on Machine Learning
The reliability of a dialogue structure coding scheme
Computational Linguistics
Towards developing general models of usability with PARADISE
Natural Language Engineering
PARADISE: a framework for evaluating spoken dialogue agents
ACL '98 Proceedings of the 35th Annual Meeting of the Association for Computational Linguistics and Eighth Conference of the European Chapter of the Association for Computational Linguistics
Learning optimal dialogue strategies: a case study of a spoken dialogue agent for email
COLING '98 Proceedings of the 17th international conference on Computational linguistics - Volume 2
Identifying user corrections automatically in spoken dialogue systems
NAACL '01 Proceedings of the second meeting of the North American Chapter of the Association for Computational Linguistics on Language technologies
The Knowledge Engineering Review
The PARADISE Evaluation Framework: Issues and Findings
Computational Linguistics
Data Mining: Practical Machine Learning Tools and Techniques, Second Edition (Morgan Kaufmann Series in Data Management Systems)
ACL '04 Proceedings of the 42nd Annual Meeting on Association for Computational Linguistics
Implications for generating clarification requests in task-oriented dialogues
ACL '05 Proceedings of the 43rd Annual Meeting on Association for Computational Linguistics
Partially observable Markov decision processes for spoken dialog systems
Computer Speech and Language
Using machine learning to explore human multimodal clarification strategies
COLING-ACL '06 Proceedings of the COLING/ACL on Main conference poster sessions
Hybrid reinforcement/supervised learning of dialogue policies from fixed data sets
Computational Linguistics
Data-driven user simulation for automated evaluation of spoken dialog systems
Computer Speech and Language
Automatic annotation of context and speech acts for dialogue corpora
Natural Language Engineering
NAACL-HLT-Dialog '07 Proceedings of the Workshop on Bridging the Gap: Academic and Industrial Research in Dialog Technologies
User simulations for context-sensitive speech recognition in spoken dialogue systems
EACL '09 Proceedings of the 12th Conference of the European Chapter of the Association for Computational Linguistics
Natural language generation as planning under uncertainty for spoken dialogue systems
EACL '09 Proceedings of the 12th Conference of the European Chapter of the Association for Computational Linguistics
Comparing user simulation models for dialog strategy learning
NAACL-Short '07 Human Language Technologies 2007: The Conference of the North American Chapter of the Association for Computational Linguistics; Companion Volume, Short Papers
Agenda-based user simulation for bootstrapping a POMDP dialogue system
NAACL-Short '07 Human Language Technologies 2007: The Conference of the North American Chapter of the Association for Computational Linguistics; Companion Volume, Short Papers
The Hidden Information State model: A practical framework for POMDP-based spoken dialogue management
Computer Speech and Language
Degrees of grounding based on evidence of understanding
SIGdial '08 Proceedings of the 9th SIGdial Workshop on Discourse and Dialogue
Journal of Artificial Intelligence Research
Optimizing dialogue management with reinforcement learning: experiments with the NJFun system
Journal of Artificial Intelligence Research
Learning human multimodal dialogue strategies
Natural Language Engineering
The Knowledge Engineering Review
Learning to adapt to unknown users: referring expression generation in spoken dialogue systems
ACL '10 Proceedings of the 48th Annual Meeting of the Association for Computational Linguistics
Optimising information presentation for spoken dialogue systems
ACL '10 Proceedings of the 48th Annual Meeting of the Association for Computational Linguistics
A probabilistic framework for dialog simulation and optimal strategy learning
IEEE Transactions on Audio, Speech, and Language Processing
Natural language generation as planning under uncertainty for spoken dialogue systems
Empirical methods in natural language generation
Introduction to special issue on machine learning for adaptivity in spoken dialogue systems
ACM Transactions on Speech and Language Processing (TSLP)
SIGDIAL '11 Proceedings of the SIGDIAL 2011 Conference
Adaptive information presentation for spoken dialogue systems: evaluation with human subjects
ENLG '11 Proceedings of the 13th European Workshop on Natural Language Generation
After dialog went pervasive: separating dialog behavior modeling and task modeling
SDCTD '12 NAACL-HLT Workshop on Future Directions and Needs in the Spoken Dialog Community: Tools and Data
Reward shaping for statistical optimisation of dialogue management
SLSP'13 Proceedings of the First international conference on Statistical Language and Speech Processing
Hi-index | 0.00 |
We present a new data-driven methodology for simulation-based dialogue strategy learning, which allows us to address several problems in the field of automatic optimization of dialogue strategies: learning effective dialogue strategies when no initial data or system exists, and determining a data-driven reward function. In addition, we evaluate the result with real users, and explore how results transfer between simulated and real interactions. We use Reinforcement Learning (RL) to learn multimodal dialogue strategies by interaction with a simulated environment which is "bootstrapped" from small amounts of Wizard-of-Oz (WOZ) data. This use of WOZ data allows data-driven development of optimal strategies for domains where no working prototype is available. Using simulation-based RL allows us to find optimal policies which are not (necessarily) present in the original data. Our results show that simulation-based RL significantly outperforms the average (human wizard) strategy as learned from the data by using Supervised Learning. The bootstrapped RL-based policy gains on average 50 times more reward when tested in simulation, and almost 18 times more reward when interacting with real users. Users also subjectively rate the RL-based policy on average 10% higher. We also show that results from simulated interaction do transfer to interaction with real users, and we explicitly evaluate the stability of the data-driven reward function.