Attention, intentions, and the structure of discourse
Computational Linguistics
A proposal for incremental dialogue evaluation
HLT '91 Proceedings of the workshop on Speech and Natural Language
Beyond class A: a proposal for automatic evaluation of discourse
HLT '90 Proceedings of the workshop on Speech and Natural Language
User Interfaces for voice applications
Voice communication between humans and machines
Empirical methods for artificial intelligence
Empirical methods for artificial intelligence
Assessing agreement on classification tasks: the kappa statistic
Computational Linguistics
The effect of resource limits and task complexity on collaborative planning in dialogue
Artificial Intelligence - Special volume on empirical methods
Development Principles for Dialog-Based Interfaces
ECAI '96 Workshop on Dialogue Processing in Spoken Language Systems
Discourse segmentation by human and automated means
Computational Linguistics
Computational Linguistics
ACL '84 Proceedings of the 10th International Conference on Computational Linguistics and 22nd annual meeting on Association for Computational Linguistics
Response generation in collaborative negotiation
ACL '95 Proceedings of the 33rd annual meeting on Association for Computational Linguistics
A prosodic analysis of discourse segments in direction-giving monologues
ACL '96 Proceedings of the 34th annual meeting on Association for Computational Linguistics
Estimating upper and lower bounds on the performance of word-sense disambiguation programs
ACL '92 Proceedings of the 30th annual meeting on Association for Computational Linguistics
Taking the initiative in natural language data base interactions: justifying why
COLING '82 Proceedings of the 9th conference on Computational linguistics - Volume 1
Experiments in evaluating interactive spoken language systems
HLT '91 Proceedings of the workshop on Speech and Natural Language
HLT '91 Proceedings of the workshop on Speech and Natural Language
What can I say?: evaluating a spoken language interface to Email
Proceedings of the SIGCHI Conference on Human Factors in Computing Systems
Intelligent speech for information systems: towards biliteracy and trilingualism
CUU '00 Proceedings on the 2000 conference on Universal Usability
A comparative study of speech in the call center: natural language call routing vs. touch-tone menus
Proceedings of the SIGCHI Conference on Human Factors in Computing Systems
Spoken dialogue technology: enabling the conversational user interface
ACM Computing Surveys (CSUR)
An Approach to Mixed Initiative Spoken Information Retrieval Dialogue
User Modeling and User-Adapted Interaction
Designing and Evaluating an Adaptive Spoken Dialogue System
User Modeling and User-Adapted Interaction
Automated testing of mySAP business processes: efficient usage of the SAP test workbench
Software quality and software testing in internet times
A User-Sensitive Spoken Dialogue System Incorporating Emotional Responsiveness
SETN '02 Proceedings of the Second Hellenic Conference on AI: Methods and Applications of Artificial Intelligence
A User Modeling Approach to Determining System Initiative in Mixed-Initiative AI Systems
UM '01 Proceedings of the 8th International Conference on User Modeling 2001
Mixed-Initiative Translation of Web Pages
AMTA '00 Proceedings of the 4th Conference of the Association for Machine Translation in the Americas on Envisioning Machine Translation in the Information Future
Conversational interface technologies
The human-computer interaction handbook
Contextually appropriate reference generation
Natural Language Engineering
Towards developing general models of usability with PARADISE
Natural Language Engineering
MIMIC: an adaptive mixed initiative spoken dialogue system for information queries
ANLC '00 Proceedings of the sixth conference on Applied natural language processing
Evaluating automatic dialogue strategy adaptation for a spoken dialogue system
NAACL 2000 Proceedings of the 1st North American chapter of the Association for Computational Linguistics conference
Evaluating response strategies in a Web-based spoken dialogue agent
COLING '98 Proceedings of the 17th international conference on Computational linguistics - Volume 2
Learning optimal dialogue strategies: a case study of a spoken dialogue agent for email
COLING '98 Proceedings of the 17th international conference on Computational linguistics - Volume 2
Computational Modeling and Analysis of Knowledge Sharing in Collaborative Distance Learning
User Modeling and User-Adapted Interaction
Evaluating discourse understanding in spoken dialogue systems
ACM Transactions on Speech and Language Processing (TSLP)
Two-way adaptation for robust input interpretation in practical multimodal conversation systems
Proceedings of the 10th international conference on Intelligent user interfaces
A three-tiered evaluation approach for interactive spoken dialogue systems
HLT '01 Proceedings of the first international conference on Human language technology research
Empirical methods for evaluating dialog systems
ELDS '01 Proceedings of the workshop on Evaluation for Language and Dialogue Systems - Volume 9
Verification and validation of language processing systems: is it evaluation?
ELDS '01 Proceedings of the workshop on Evaluation for Language and Dialogue Systems - Volume 9
An empirical study of speech recognition errors in a task-oriented dialogue system
SIGDIAL '01 Proceedings of the Second SIGdial Workshop on Discourse and Dialogue - Volume 16
Empirical methods for evaluating dialog systems
SIGDIAL '01 Proceedings of the Second SIGdial Workshop on Discourse and Dialogue - Volume 16
A new taxonomy for the quality of telephone services based on spoken dialogue systems
SIGDIAL '02 Proceedings of the 3rd SIGdial workshop on Discourse and dialogue - Volume 2
Training a Dialogue Act Tagger for human-human and human-computer travel dialogues
SIGDIAL '02 Proceedings of the 3rd SIGdial workshop on Discourse and dialogue - Volume 2
Evaluation metrics for generation
INLG '00 Proceedings of the first international conference on Natural language generation - Volume 14
The PARADISE Evaluation Framework: Issues and Findings
Computational Linguistics
Using model trees for evaluating dialog error conditions based on acoustic information
Proceedings of the 1st ACM international workshop on Human-centered multimedia
HLT-NAACL '06 Proceedings of the main conference on Human Language Technology Conference of the North American Chapter of the Association of Computational Linguistics
Adapting the interaction in a call centre system
Interacting with Computers
Partially observable Markov decision processes for spoken dialog systems
Computer Speech and Language
Segmented and unsegmented dialogue-act annotation with statistical dialogue models
COLING-ACL '06 Proceedings of the COLING/ACL on Main conference poster sessions
Stochastic discourse modeling in spoken dialogue systems using semantic dependency graphs
COLING-ACL '06 Proceedings of the COLING/ACL on Main conference poster sessions
Automatic learning of dialogue strategy using dialogue simulation and reinforcement learning
HLT '02 Proceedings of the second international conference on Human Language Technology Research
Testing the performance of spoken dialogue systems by means of an artificially simulated user
Artificial Intelligence Review
Implementation of DomCAT: the domain complexity analysis tool for natural language dialog processing
TAPIA '07 Proceedings of the 2007 conference on Diversity in computing
Predicting the quality and usability of spoken dialogue services
Speech Communication
Bootstrapping spoken dialogue systems by exploiting reusable libraries
Natural Language Engineering
Handling out of domain topics by a conversational character
Proceedings of the 3rd international conference on Digital Interactive Media in Entertainment and Arts
Data-Driven Dialogue for Interactive Question Answering
AI*IA '07 Proceedings of the 10th Congress of the Italian Association for Artificial Intelligence on AI*IA 2007: Artificial Intelligence and Human-Oriented Computing
Weighting the Coefficients in PARADISE Models to Increase Their Generalizability
PIT '08 Proceedings of the 4th IEEE tutorial and research workshop on Perception and Interactive Technologies for Speech-Based Systems: Perception in Multimodal Dialogue Systems
Towards a Perception-Based Evaluation Model for Spoken Dialogue Systems
PIT '08 Proceedings of the 4th IEEE tutorial and research workshop on Perception and Interactive Technologies for Speech-Based Systems: Perception in Multimodal Dialogue Systems
Evaluation of the Slovak Spoken Dialogue System Based on ITU-T
TSD '08 Proceedings of the 11th international conference on Text, Speech and Dialogue
Embodied conversational agents for voice-biometric interfaces
ICMI '08 Proceedings of the 10th international conference on Multimodal interfaces
Example-based dialog modeling for practical multi-domain dialog system
Speech Communication
The RavenClaw dialog management framework: Architecture and systems
Computer Speech and Language
Call browser: a system to improve the caller experience by analyzing live calls end-to-end
Proceedings of the SIGCHI Conference on Human Factors in Computing Systems
Questionnaires for eliciting evaluation data from users of interactive question answering systems
Natural Language Engineering
A flexible dialogue system for enhancing web usability
Proceedings of the 18th international conference on World wide web
NAACL-HLT-Dialog '07 Proceedings of the Workshop on Bridging the Gap: Academic and Industrial Research in Dialog Technologies
AdaRTE: an extensible and adaptable architecture for dialog systems
NAACL-HLT-Dialog '07 Proceedings of the Workshop on Bridging the Gap: Academic and Industrial Research in Dialog Technologies
WIRE: a wearable spoken language understanding system for the military
NAACL-HLT-Dialog '07 Proceedings of the Workshop on Bridging the Gap: Academic and Industrial Research in Dialog Technologies
Comparing Linguistic Features for Modeling Learning in Computer Tutoring
Proceedings of the 2007 conference on Artificial Intelligence in Education: Building Technology Rich Learning Contexts That Work
Analysis of a new simulation approach to dialog system evaluation
Speech Communication
Computer Speech and Language
Spoken language understanding using weakly supervised learning
Computer Speech and Language
User simulation as testing for spoken dialog systems
SIGdial '08 Proceedings of the 9th SIGdial Workshop on Discourse and Dialogue
A framework for model-based evaluation of spoken dialog systems
SIGdial '08 Proceedings of the 9th SIGdial Workshop on Discourse and Dialogue
Journal of Artificial Intelligence Research
Automatically training a problematic dialogue predictor for a spoken dialogue system
Journal of Artificial Intelligence Research
Setting up an evaluation infrastructure for human language technologies in Europe
Evalinitiatives '03 Proceedings of the EACL 2003 Workshop on Evaluation Initiatives in Natural Language Processing: are evaluation methods, metrics and resources reusable?
ISDS '97 Interactive Spoken Dialog Systems on Bringing Speech and NLP Together in Real Applications
Adaptive help for speech dialogue systems based on learning and forgetting of speech commands
SigDIAL '06 Proceedings of the 7th SIGdial Workshop on Discourse and Dialogue
Evaluating description and reference strategies in a cooperative human-robot dialogue system
IJCAI'09 Proceedings of the 21st international jont conference on Artifical intelligence
Comparing objective and subjective measures of usability in a human-robot dialogue system
ACL '09 Proceedings of the Joint Conference of the 47th Annual Meeting of the ACL and the 4th International Joint Conference on Natural Language Processing of the AFNLP: Volume 2 - Volume 2
INLG '08 Proceedings of the Fifth International Natural Language Generation Conference
Modeling user satisfaction with Hidden Markov Model
SIGDIAL '09 Proceedings of the SIGDIAL 2009 Conference: The 10th Annual Meeting of the Special Interest Group on Discourse and Dialogue
Towards a method for evaluating naturalness in conversational dialog systems
SMC'09 Proceedings of the 2009 IEEE international conference on Systems, Man and Cybernetics
The Knowledge Engineering Review
Usability in location-based services: context and mobile map navigation
UAHCI'07 Proceedings of the 4th international conference on Universal access in human-computer interaction: ambient interaction
A framework for evaluating the usability of spoken language dialog systems (SLDSs)
UI-HCII'07 Proceedings of the 2nd international conference on Usability and internationalization
A wizard-of-Oz system evaluation study
TSD'07 Proceedings of the 10th international conference on Text, speech and dialogue
Human robot interactions: towards the implementation of adaptive strategies for robust communication
BVAI'07 Proceedings of the 2nd international conference on Advances in brain, vision and artificial intelligence
Towards an objective test for meeting browsers: the BET4TQB pilot experiment
MLMI'07 Proceedings of the 4th international conference on Machine learning for multimodal interaction
HIV health information access using spoken dialogue systems: touchtone vs. speech
ICTD'09 Proceedings of the 3rd international conference on Information and communication technologies and development
Designing and evaluating a wizarded uncertainty-adaptive spoken dialogue tutoring system
Computer Speech and Language
Evaluating multimodal systems: a comparison of established questionnaires and interaction parameters
Proceedings of the 6th Nordic Conference on Human-Computer Interaction: Extending Boundaries
Multimodal and mobile conversational Health and Fitness Companions
Computer Speech and Language
The first challenge on generating instructions in virtual environments
Empirical methods in natural language generation
User and noise adaptive dialogue management using hybrid system actions
IWSDS'10 Proceedings of the Second international conference on Spoken dialogue systems for ambient environments
IWSDS'10 Proceedings of the Second international conference on Spoken dialogue systems for ambient environments
A user model to predict user satisfaction with spoken dialog systems
IWSDS'10 Proceedings of the Second international conference on Spoken dialogue systems for ambient environments
A study toward an evaluation method for spoken dialogue systems considering user criteria
IWSDS'10 Proceedings of the Second international conference on Spoken dialogue systems for ambient environments
Modeling user satisfaction transitions in dialogues from overall ratings
SIGDIAL '10 Proceedings of the 11th Annual Meeting of the Special Interest Group on Discourse and Dialogue
Sparse approximate dynamic programming for dialog management
SIGDIAL '10 Proceedings of the 11th Annual Meeting of the Special Interest Group on Discourse and Dialogue
Computational Linguistics
Specification and evaluation of a Spanish conversational system using dialogue models
IBERAMIA'10 Proceedings of the 12th Ibero-American conference on Advances in artificial intelligence
Development of a tour-guide robot using dialogue models and a cognitive architecture
IBERAMIA'10 Proceedings of the 12th Ibero-American conference on Advances in artificial intelligence
Sample-efficient batch reinforcement learning for dialogue management optimization
ACM Transactions on Speech and Language Processing (TSLP)
Modeling spoken decision support dialogue and optimization of its dialogue strategy
ACM Transactions on Speech and Language Processing (TSLP)
HLT '11 Proceedings of the 49th Annual Meeting of the Association for Computational Linguistics: Human Language Technologies: short papers - Volume 2
A multitasking approach to adaptive spoken dialogue management
UAHCI'11 Proceedings of the 6th international conference on Universal access in human-computer interaction: applications and services - Volume Part IV
Machine learning for spoken dialogue management: an experiment with speech-based database querying
AIMSA'06 Proceedings of the 12th international conference on Artificial Intelligence: methodology, Systems, and Applications
SIGDIAL '11 Proceedings of the SIGDIAL 2011 Conference
Towards effective, efficient and elderly-friendly multimodal interaction
Proceedings of the 4th International Conference on PErvasive Technologies Related to Assistive Environments
Interaction strategies for an affective conversational agent
Presence: Teleoperators and Virtual Environments
Talking with robots about objects: a system-level evaluation in HRI
HRI '12 Proceedings of the seventh annual ACM/IEEE international conference on Human-Robot Interaction
Learning automata-based approach to learn dialogue policies in large state space
International Journal of Intelligent Information and Database Systems
Voice control in smart homes using distant microphones: a VoiceXML-Based approach
AmI'11 Proceedings of the Second international conference on Ambient Intelligence
ENLG '11 Proceedings of the 13th European Workshop on Natural Language Generation
System Personality and Persuasion in Human-Computer Dialogue
ACM Transactions on Interactive Intelligent Systems (TiiS)
Sample efficient on-line learning of optimal dialogue policies with kalman temporal differences
IJCAI'11 Proceedings of the Twenty-Second international joint conference on Artificial Intelligence - Volume Volume Three
International Journal of Artificial Intelligence in Education - Special issue on Best of ITS 2010
Comparing HMMs and Bayesian networks for surface realisation
NAACL HLT '12 Proceedings of the 2012 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies
An integrated development environment for spoken dialogue systems
Proceedings of the COLING-2000 Workshop on Using Toolsets and Architectures To Build NLP Systems
My automated conversation helper (MACH): helping people improve social skills.
Proceedings of the 14th ACM international conference on Multimodal interaction
SDCTD '12 NAACL-HLT Workshop on Future Directions and Needs in the Spoken Dialog Community: Tools and Data
SDCTD '12 NAACL-HLT Workshop on Future Directions and Needs in the Spoken Dialog Community: Tools and Data
A regression-based approach to modeling addressee backchannels
SIGDIAL '12 Proceedings of the 13th Annual Meeting of the Special Interest Group on Discourse and Dialogue
A Dynamic Spoken Dialogue Interface for Ambient Intelligence Interaction
International Journal of Ambient Computing and Intelligence
Factors influencing user feedback on predicted satisfaction with software systems
REFSQ'13 Proceedings of the 19th international conference on Requirements Engineering: Foundation for Software Quality
Inverse reinforcement learning for interactive systems
Proceedings of the 2nd Workshop on Machine Learning for Interactive Systems: Bridging the Gap Between Perception, Action and Communication
Social signal and user adaptation in reinforcement learning-based dialogue management
Proceedings of the 2nd Workshop on Machine Learning for Interactive Systems: Bridging the Gap Between Perception, Action and Communication
Chorus: a crowd-powered conversational assistant
Proceedings of the 26th annual ACM symposium on User interface software and technology
Reward shaping for statistical optimisation of dialogue management
SLSP'13 Proceedings of the First international conference on Statistical Language and Speech Processing
A new benchmark dataset with production methodology for short text semantic similarity algorithms
ACM Transactions on Speech and Language Processing (TSLP)
An approach to conversational agent design using semantic sentence similarity
Applied Intelligence
| Hi-index | 0.00 |
This paper presents PARADISE (PARAdigm for DIalogue System Evaluation), a general framework for evaluating spoken dialogue agents. The framework decouples task requirements from an agent's dialogue behaviors, supports comparisons among dialogue strategies, enables the calculation of performance over subdialogues and whole dialogues, specifies the relative contribution of various factors to performance, and makes it possible to compare agents performing different tasks by normalizing for task complexity.