Introduction to Reinforcement Learning
Introduction to Reinforcement Learning
Autonomous Agents and Multi-Agent Systems
Empirical Evaluation of a Reinforcement Learning Spoken Dialogue System
Proceedings of the Seventeenth National Conference on Artificial Intelligence and Twelfth Conference on Innovative Applications of Artificial Intelligence
Information state and dialogue management in the TRINDI dialogue move engine toolkit
Natural Language Engineering
Towards developing general models of usability with PARADISE
Natural Language Engineering
Kernel Methods for Pattern Analysis
Kernel Methods for Pattern Analysis
Learning optimal dialogue strategies: a case study of a spoken dialogue agent for email
COLING '98 Proceedings of the 17th international conference on Computational linguistics - Volume 2
DATE: a dialogue act tagging scheme for evaluation of spoken dialogue systems
HLT '01 Proceedings of the first international conference on Human language technology research
Quantitative and qualitative evaluation of Darpa Communicator spoken dialogue systems
ACL '01 Proceedings of the 39th Annual Meeting on Association for Computational Linguistics
Spoken dialogue management using probabilistic reasoning
ACL '00 Proceedings of the 38th Annual Meeting on Association for Computational Linguistics
The Knowledge Engineering Review
ACL '04 Proceedings of the 42nd Annual Meeting on Association for Computational Linguistics
Learning more effective dialogue strategies using limited dialogue move features
ACL-44 Proceedings of the 21st International Conference on Computational Linguistics and the 44th annual meeting of the Association for Computational Linguistics
Fast reinforcement learning of dialog strategies
ICASSP '00 Proceedings of the Acoustics, Speech, and Signal Processing, 2000. on IEEE International Conference - Volume 02
Residential computer usage patterns in Japan and associated life cycle energy use
ISEE '05 Proceedings of the International Symposium on Electronics and the Environment
Using machine learning to explore human multimodal clarification strategies
COLING-ACL '06 Proceedings of the COLING/ACL on Main conference poster sessions
Automatic learning of dialogue strategy using dialogue simulation and reinforcement learning
HLT '02 Proceedings of the second international conference on Human Language Technology Research
EACL '06 Proceedings of the Eleventh Conference of the European Chapter of the Association for Computational Linguistics: Posters & Demonstrations
Optimizing dialogue management with reinforcement learning: experiments with the NJFun system
Journal of Artificial Intelligence Research
Fast reinforcement learning of dialogue policies using stable function approximation
IJCNLP'04 Proceedings of the First international joint conference on Natural Language Processing
A probabilistic framework for dialog simulation and optimal strategy learning
IEEE Transactions on Audio, Speech, and Language Processing
Special issue on interactive question answering: introduction
Natural Language Engineering
Automatic annotation of context and speech acts for dialogue corpora
Natural Language Engineering
Improving recommender systems with adaptive conversational strategies
Proceedings of the 20th ACM conference on Hypertext and hypermedia
Natural language generation as planning under uncertainty for spoken dialogue systems
EACL '09 Proceedings of the 12th Conference of the European Chapter of the Association for Computational Linguistics
Evaluation of a hierarchical reinforcement learning spoken dialogue system
Computer Speech and Language
Hybrid approach to user intention modeling for dialog simulation
ACLShort '09 Proceedings of the ACL-IJCNLP 2009 Conference Short Papers
Evaluating the effectiveness of information presentation in a full end-to-end dialogue system
SIGDIAL '09 Proceedings of the SIGDIAL 2009 Conference: The 10th Annual Meeting of the Special Interest Group on Discourse and Dialogue
Learning human multimodal dialogue strategies
Natural Language Engineering
The Knowledge Engineering Review
Leveraging hidden dialogue state to select tutorial moves
IUNLPBEA '10 Proceedings of the NAACL HLT 2010 Fifth Workshop on Innovative Use of NLP for Building Educational Applications
Using reinforcement learning to create communication channel management strategies for diverse users
SLPAT '10 Proceedings of the NAACL HLT 2010 Workshop on Speech and Language Processing for Assistive Technologies
Harvesting re-usable high-level rules for expository dialogue generation
INLG '10 Proceedings of the 6th International Natural Language Generation Conference
Hybrid user intention modeling to diversify dialog simulations
Computer Speech and Language
Natural language generation as planning under uncertainty for spoken dialogue systems
Empirical methods in natural language generation
Sparse approximate dynamic programming for dialog management
SIGDIAL '10 Proceedings of the 11th Annual Meeting of the Special Interest Group on Discourse and Dialogue
Introduction to special issue on machine learning for adaptivity in spoken dialogue systems
ACM Transactions on Speech and Language Processing (TSLP)
Spatially-aware dialogue control using hierarchical reinforcement learning
ACM Transactions on Speech and Language Processing (TSLP)
Sample-efficient batch reinforcement learning for dialogue management optimization
ACM Transactions on Speech and Language Processing (TSLP)
Classifying dialogue in high-dimensional space
ACM Transactions on Speech and Language Processing (TSLP)
Sample efficient on-line learning of optimal dialogue policies with kalman temporal differences
IJCAI'11 Proceedings of the Twenty-Second international joint conference on Artificial Intelligence - Volume Volume Three
Optimising incremental dialogue decisions using information density for interactive systems
EMNLP-CoNLL '12 Proceedings of the 2012 Joint Conference on Empirical Methods in Natural Language Processing and Computational Natural Language Learning
An unsupervised approach to user simulation: toward self-improving dialog systems
SIGDIAL '12 Proceedings of the 13th Annual Meeting of the Special Interest Group on Discourse and Dialogue
Probabilistic dialogue models with prior domain knowledge
SIGDIAL '12 Proceedings of the 13th Annual Meeting of the Special Interest Group on Discourse and Dialogue
Exploiting machine-transcribed dialog corpus to improve multiple dialog states tracking methods
SIGDIAL '12 Proceedings of the 13th Annual Meeting of the Special Interest Group on Discourse and Dialogue
Engineering Applications of Artificial Intelligence
Gaussian Processes for POMDP-Based Dialogue Manager Optimization
IEEE/ACM Transactions on Audio, Speech and Language Processing (TASLP)
| Hi-index | 0.00 |
We propose a method for learning dialogue management policies from a fixed data set. The method addresses the challenges posed by Information State Update (ISU)-based dialogue systems, which represent the state of a dialogue as a large set of features, resulting in a very large state space and a huge policy space. To address the problem that any fixed data set will only provide information about small portions of these state and policy spaces, we propose a hybrid model that combines reinforcement learning with supervised learning. The reinforcement learning is used to optimize a measure of dialogue reward, while the supervised learning is used to restrict the learned policy to the portions of these spaces for which we have data. We also use linear function approximation to address the need to generalize from a fixed amount of data to large state spaces. To demonstrate the effectiveness of this method on this challenging task, we trained this model on the COMMUNICATOR corpus, to which we have added annotations for user actions and Information States. When tested with a user simulation trained on a different part of the same data set, our hybrid model outperforms a pure supervised learning model and a pure reinforcement learning model. It also outperforms the hand-crafted systems on the COMMUNICATOR data, according to automatic evaluation measures, improving over the average COMMUNICATOR system policy by 10%. The proposed method will improve techniques for bootstrapping and automatic optimization of dialogue management policies from limited initial data sets.