Predicting student emotions in computer-human tutoring dialogues

Authors:
Diane J. Litman;Kate Forbes-Riley
Affiliations:
University of Pittsburgh, Pittsburgh, PA;University of Pittsburgh, Pittsburgh, PA
Venue:
ACL '04 Proceedings of the 42nd Annual Meeting on Association for Computational Linguistics
Year:
2004

Citing 10
Cited 46

Data mining: practical machine learning tools and techniques with Java implementations

Data mining: practical machine learning tools and techniques with Java implementations
Intelligent tutoring systems with conversational dialogue

AI Magazine
The Architecture of Why2-Atlas: A Coach for Qualitative Physics Essay Writing

ITS '02 Proceedings of the 6th International Conference on Intelligent Tutoring Systems
Adding Human-Provided Emotional Scaffolding to an Automated Reading Tutor That Listens Increases Student Persistence

ITS '02 Proceedings of the 6th International Conference on Intelligent Tutoring Systems
How to find trouble in communication

Speech Communication - Special issue on speech and emotion
The production and recognition of emotions in speech: features and algorithms

International Journal of Human-Computer Studies - Application of affective computing in human—Computer interaction
An Affective Model of Interplay between Emotions and Learning: Reengineering Educational Pedagogy-Building a Learning Companion

ICALT '01 Proceedings of the IEEE International Conference on Advanced Learning Technologies
Emotion detection in task-oriented spoken dialogues

ICME '03 Proceedings of the 2003 International Conference on Multimedia and Expo - Volume 3 (ICME '03) - Volume 03
Co-training for predicting emotions with spoken dialogue data

ACLdemo '04 Proceedings of the ACL 2004 on Interactive poster and demonstration sessions
ITSPOKE: an intelligent tutoring spoken dialogue system

HLT-NAACL--Demonstrations '04 Demonstration Papers at HLT-NAACL 2004

Inductive creation of an annotation schema for manually indexing clinical conditions from emergency department reports

Journal of Biomedical Informatics
Audio-visual emotion recognition in adult attachment interview

Proceedings of the 8th international conference on Multimodal interfaces
Emotions from text: machine learning for text-based emotion prediction

HLT '05 Proceedings of the conference on Human Language Technology and Empirical Methods in Natural Language Processing
Modelling user satisfaction and student learning in a spoken dialogue tutoring system with generic, tutoring, and user affect parameters

HLT-NAACL '06 Proceedings of the main conference on Human Language Technology Conference of the North American Chapter of the Association of Computational Linguistics
Toward an Affect-Sensitive AutoTutor

IEEE Intelligent Systems
A survey of affect recognition methods: audio, visual and spontaneous expressions

Proceedings of the 9th international conference on Multimodal interfaces
Modeling parallel and reactive empathy in virtual agents: an inductive approach

Proceedings of the 7th international joint conference on Autonomous agents and multiagent systems - Volume 1
Automatic Recognition of Emotions from Speech: A Review of the Literature and Recommendations for Practical Realisation

Affect and Emotion in Human-Computer Interaction
Self Versus Teacher Judgments of Learner Emotions During a Tutoring Session with AutoTutor

ITS '08 Proceedings of the 9th international conference on Intelligent Tutoring Systems
Automatic Recognition of Spontaneous Emotions in Speech Using Acoustic and Lexical Features

MLMI '08 Proceedings of the 5th international workshop on Machine Learning for Multimodal Interaction
Predicting Affective States expressed through an Emote-Aloud Procedure from AutoTutor's Mixed-Initiative Dialogue

International Journal of Artificial Intelligence in Education
Spoken Versus Typed Human and Computer Dialogue Tutoring

International Journal of Artificial Intelligence in Education
Responding to Student Uncertainty in Spoken Tutorial Dialogue Systems

International Journal of Artificial Intelligence in Education
Representing Emotions with Linguistic Acuity

CICLing '07 Proceedings of the 8th International Conference on Computational Linguistics and Intelligent Text Processing
Tools for Authoring a Dialogue Agent that Participates in Learning Studies

Proceedings of the 2007 conference on Artificial Intelligence in Education: Building Technology Rich Learning Contexts That Work
Getting Under the Skin of Learners: Tools for Evaluating Emotional Experience

Proceedings of the 2007 conference on Artificial Intelligence in Education: Building Technology Rich Learning Contexts That Work
Emotions and Learning with AutoTutor

Proceedings of the 2007 conference on Artificial Intelligence in Education: Building Technology Rich Learning Contexts That Work
Empirically building and evaluating a probabilistic model of user affect

User Modeling and User-Adapted Interaction
Emotion classification using massive examples extracted from the web

COLING '08 Proceedings of the 22nd International Conference on Computational Linguistics - Volume 1
Responding to Learners' Cognitive-Affective States with Supportive and Shakeup Dialogues

Proceedings of the 13th International Conference on Human-Computer Interaction. Part III: Ubiquitous and Intelligent Interaction
Humor: prosody analysis and automatic recognition for F*R*I*E*N*D*S*

EMNLP '06 Proceedings of the 2006 Conference on Empirical Methods in Natural Language Processing
Exploring affect-context dependencies for adaptive system development

NAACL-Short '07 Human Language Technologies 2007: The Conference of the North American Chapter of the Association for Computational Linguistics; Companion Volume, Short Papers
Probabilistic semantic classifier

SMO'09 Proceedings of the 9th WSEAS international conference on Simulation, modelling and optimization
Multimethod assessment of affective experience and expression during deep learning

International Journal of Learning Technology
Supporting affective communication in the classroom with the Subtle Stone

International Journal of Learning Technology
Emotion recognition using facial expressions with active appearance models

HCI '08 Proceedings of the Third IASTED International Conference on Human Computer Interaction
Better to be frustrated than bored: The incidence, persistence, and impact of learners' cognitive-affective states during interactions with three different computer-based learning environments

International Journal of Human-Computer Studies
Agent based affective tutoring systems: A pilot study

Computers & Education
Multimodal semi-automated affect detection from conversational cues, gross body language, and facial features

User Modeling and User-Adapted Interaction
Detecting emotional state of a child in a conversational computer game

Computer Speech and Language
Emotion detection in email customer care

CAAGET '10 Proceedings of the NAACL HLT 2010 Workshop on Computational Approaches to Analysis and Generation of Emotion in Text
Audio-visual spontaneous emotion recognition

ICMI'06/IJCAI'07 Proceedings of the ICMI 2006 and IJCAI 2007 international conference on Artifical intelligence for human computing
Benefits and challenges of real-time uncertainty detection and adaptation in a spoken dialogue computer tutor

Speech Communication
Adaptive machine learning approach for emotional email classification

HCII'11 Proceedings of the 14th international conference on Human-computer interaction: towards mobile and intelligent interaction environments - Volume Part III
Towards a brain-sensitive intelligent tutoring system: detecting emotions from brainwaves

Advances in Artificial Intelligence
The dynamics between student affect and behavior occurring outside of educational software

ACII'11 Proceedings of the 4th international conference on Affective computing and intelligent interaction - Volume Part I
Controlling user perceptions of linguistic style: Trainable generation of personality traits

Computational Linguistics
Affect detection from human-computer dialogue with an intelligent tutoring system

IVA'06 Proceedings of the 6th international conference on Intelligent Virtual Agents
Pain monitoring: A dynamic and context-sensitive system

Pattern Recognition
Grounding emotions in human-machine conversational systems

INTETAIN'05 Proceedings of the First international conference on Intelligent Technologies for Interactive Entertainment
Spontaneous pain expression recognition in video sequences

VoCS'08 Proceedings of the 2008 international conference on Visions of Computer Science: BCS International Academic Conference
Automatic detection of speaker state: Lexical, prosodic, and phonetic approaches to level-of-interest and intoxication classification

Computer Speech and Language
"Love ya, jerkface": using sparse log-linear models to build positive (and impolite) relationships with teens

SIGDIAL '12 Proceedings of the 13th Annual Meeting of the Special Interest Group on Discourse and Dialogue
A dynamic multimodal approach for assessing learners' interaction experience

Proceedings of the 15th ACM on International conference on multimodal interaction
Virtual butler: what can we learn from adaptive user interfaces?

Your Virtual Butler
Knowledge Elicitation Methods for Affect Modelling in Education

International Journal of Artificial Intelligence in Education

Quantified Score

Hi-index	0.00

Visualization

Abstract

We examine the utility of speech and lexical features for predicting student emotions in computer-human spoken tutoring dialogues. We first annotate student turns for negative, neutral, positive and mixed emotions. We then extract acoustic-prosodic features from the speech signal, and lexical items from the transcribed or recognized speech. We compare the results of machine learning experiments using these features alone or in combination to predict various categorizations of the annotated student emotions. Our best results yield a 19-36% relative improvement in error reduction over a baseline. Finally, we compare our results with emotion prediction in human-human tutoring dialogues.