Intelligence without representation
Artificial Intelligence
ELIZA—a computer program for the study of natural language communication between man and machine
Communications of the ACM
Alan Turing
Does the turing test demonstrate intelligence or not?
AAAI'06 proceedings of the 21st national conference on Artificial intelligence - Volume 2
Naturalness of an utterance based on the automatically retrieved commonsense
IJCAI'05 Proceedings of the 19th international joint conference on Artificial intelligence
Towards a method for evaluating naturalness in conversational dialog systems
SMC'09 Proceedings of the 2009 IEEE international conference on Systems, Man and Cybernetics
Evaluation and validation of a conversational agent embodied in a bookstore
ERCIM'02 Proceedings of the User interfaces for all 7th international conference on Universal access: theoretical perspectives, practice, and experience
| Hi-index | 0.00 |
The aim of this research is to generate measurable evaluation criteria acceptable to chatbot users. Results of two studies are summarised. In the first, fourteen participants were asked to do a critical incident analysis of their transcriptions with an ELIZA-type chatbot. Results were content analysed, and yielded seven overall themes. In the second, these themes were made into statements of an attitude-like nature, and 20 participants chatted with five winning entrants in the 2011 Chatterbox Challenge and five which failed to place. Latent variable analysis reduced the themes to four, resulting in four subscales with strong reliability which discriminated well between the two categories of chatbots. Content analysis of freeform comments led to a proposal of four dimensions along which people judge the naturalness of a conversation with chatbots.