CYC: a large-scale investment in knowledge infrastructure
Communications of the ACM
ELIZA—a computer program for the study of natural language communication between man and machine
Communications of the ACM
Embodied agents for multi-party dialogue in immersive virtual worlds
Proceedings of the first international joint conference on Autonomous agents and multiagent systems: part 2
An Empirical Evaluation of a System for Text Knowledge Acquisition
EKAW '97 Proceedings of the 10th European Workshop on Knowledge Acquisition, Modeling and Management
PARADISE: a framework for evaluating spoken dialogue agents
ACL '98 Proceedings of the 35th Annual Meeting of the Association for Computational Linguistics and Eighth Conference of the European Chapter of the Association for Computational Linguistics
A three-tiered evaluation approach for interactive spoken dialogue systems
HLT '01 Proceedings of the first international conference on Human language technology research
Semantic coherence scoring using an ontology
NAACL '03 Proceedings of the 2003 Conference of the North American Chapter of the Association for Computational Linguistics on Human Language Technology - Volume 1
Evaluating mass knowledge acquisition using the ALICE chatterbot: the AZ-ALICE dialog system
International Journal of Human-Computer Studies
Effect of modality on collaboration with a dialogue system
International Journal of Human-Computer Studies
Different measurements metrics to evaluate a chatbot system
NAACL-HLT-Dialog '07 Proceedings of the Workshop on Bridging the Gap: Academic and Industrial Research in Dialog Technologies
Naturalness of an utterance based on the automatically retrieved commonsense
IJCAI'05 Proceedings of the 19th international joint conference on Artificial intelligence
Evaluation and validation of a conversational agent embodied in a bookstore
ERCIM'02 Proceedings of the User interfaces for all 7th international conference on Universal access: theoretical perspectives, practice, and experience
An Oz-centric review of interactive drama and believable agents
Artificial intelligence today
'Realness' in chatbots: establishing quantifiable criteria
HCI'13 Proceedings of the 15th international conference on Human-Computer Interaction: interaction modalities and techniques - Volume Part IV
A Procedure to Create a Pedagogic Conversational Agent in Secondary Physics and Chemistry Education
International Journal of Information and Communication Technology Education
| Hi-index | 0.00 |
The evaluation of conversational dialog systems has remained a controversial topic, as it is challenging to quantitatively assess how well a conversation agent performs, or how much better one is compared to another. Furthermore, one of the hurdles which remains elusive in this quandary is the definition of naturalness, as demonstrated by how well a dialog system can maintain a natural conversation flow devoid of perceived awkwardness. As a step towards defining the dimensions of effectiveness and naturalness in a dialog system, this paper identifies existing evaluation practices which are then expanded to develop a more suitable assessment vehicle. This method is then applied to the LifeLike virtual avatar project.