The Architecture of Why2-Atlas: A Coach for Qualitative Physics Essay Writing
ITS '02 Proceedings of the 6th International Conference on Intelligent Tutoring Systems
Informing the Detection of the Students' Motivational State: An Empirical Study
ITS '02 Proceedings of the 6th International Conference on Intelligent Tutoring Systems
Dialogue act modeling for automatic tagging and recognition of conversational speech
Computational Linguistics
Multimodal affect recognition in learning environments
Proceedings of the 13th annual ACM international conference on Multimedia
HLT-NAACL '06 Proceedings of the main conference on Human Language Technology Conference of the North American Chapter of the Association of Computational Linguistics
Automatic detection of learner's affect from conversational cues
User Modeling and User-Adapted Interaction
Diagnosing and acting on student affect: the tutor's perspective
User Modeling and User-Adapted Interaction
The relative impact of student affect on performance models in a spoken dialogue tutoring system
User Modeling and User-Adapted Interaction
Developing a generalizable detector of when students game the system
User Modeling and User-Adapted Interaction
What Are You Feeling? Investigating Student Affective States During Expert Human Tutoring Sessions
ITS '08 Proceedings of the 9th international conference on Intelligent Tutoring Systems
Engagement tracing: using response times to model student disengagement
Proceedings of the 2005 conference on Artificial Intelligence in Education: Supporting Learning through Intelligent and Socially Informed Technology
Effects of Dissuading Unnecessary Help Requests While Providing Proactive Help
Proceedings of the 2005 conference on Artificial Intelligence in Education: Supporting Learning through Intelligent and Socially Informed Technology
Tools for Authoring a Dialogue Agent that Participates in Learning Studies
Proceedings of the 2007 conference on Artificial Intelligence in Education: Building Technology Rich Learning Contexts That Work
Repairing Disengagement With Non-Invasive Interventions
Proceedings of the 2007 conference on Artificial Intelligence in Education: Building Technology Rich Learning Contexts That Work
Empirically building and evaluating a probabilistic model of user affect
User Modeling and User-Adapted Interaction
Discourse structure and performance analysis: beyond the correlation
SIGDIAL '09 Proceedings of the SIGDIAL 2009 Conference: The 10th Annual Meeting of the Special Interest Group on Discourse and Dialogue
Models for multiparty engagement in open-world dialog
SIGDIAL '09 Proceedings of the SIGDIAL 2009 Conference: The 10th Annual Meeting of the Special Interest Group on Discourse and Dialogue
SIGDIAL '11 Proceedings of the SIGDIAL 2011 Conference
Exploring user satisfaction in a tutorial dialogue system
SIGDIAL '11 Proceedings of the SIGDIAL 2011 Conference
Adapting to when students game an intelligent tutoring system
ITS'06 Proceedings of the 8th international conference on Intelligent Tutoring Systems
Prevention of off-task gaming behavior in intelligent tutoring systems
ITS'06 Proceedings of the 8th international conference on Intelligent Tutoring Systems
NAACL HLT '12 Proceedings of the 2012 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies
Adapting to multiple affective states in spoken dialogue
SIGDIAL '12 Proceedings of the 13th Annual Meeting of the Special Interest Group on Discourse and Dialogue
Hi-index | 0.00 |
In this paper we investigate how student disengagement relates to two performance metrics in a spoken dialog computer tutoring corpus, both when disengagement is measured through manual annotation by a trained human judge, and also when disengagement is measured through automatic annotation by the system based on a machine learning model. First, we investigate whether manually labeled overall disengagement and six different disengagement types are predictive of learning and user satisfaction in the corpus. Our results show that although students' percentage of overall disengaged turns negatively correlates both with the amount they learn and their user satisfaction, the individual types of disengagement correlate differently: some negatively correlate with learning and user satisfaction, while others don't correlate with either metric at all. Moreover, these relationships change somewhat depending on student prerequisite knowledge level. Furthermore, using multiple disengagement types to predict learning improves predictive power. Overall, these manual label-based results suggest that although adapting to disengagement should improve both student learning and user satisfaction in computer tutoring, maximizing performance requires the system to detect and respond differently based on disengagement type. Next, we present an approach to automatically detecting and responding to user disengagement types based on their differing correlations with correctness. Investigation of our machine learning model of user disengagement shows that its automatic labels negatively correlate with both performance metrics in the same way as the manual labels. The similarity of the correlations across the manual and automatic labels suggests that the automatic labels are a reasonable substitute for the manual labels. Moreover, the significant negative correlations themselves suggest that redesigning ITSPOKE to automatically detect and respond to disengagement has the potential to remediate disengagement and thereby improve performance, even in the presence of noise introduced by the automatic detection process.