Predicting facial indicators of confusion with hidden Markov models

  • Authors:
  • Joseph F. Grafsgaard;Kristy Elizabeth Boyer;James C. Lester

  • Affiliations:
  • Department of Computer Science, North Carolina State University, Raleigh, North Carolina;Department of Computer Science, North Carolina State University, Raleigh, North Carolina;Department of Computer Science, North Carolina State University, Raleigh, North Carolina

  • Venue:
  • ACII'11 Proceedings of the 4th international conference on Affective computing and intelligent interaction - Volume Part I
  • Year:
  • 2011

Quantified Score

Hi-index 0.00

Visualization

Abstract

Affect plays a vital role in learning. During tutoring, particular affective states may benefit or detract from student learning. A key cognitiveaffective state is confusion, which has been positively associated with effective learning. Although identifying episodes of confusion presents significant challenges, recent investigations have identified correlations between confusion and specific facial movements. This paper builds on those findings to create a predictive model of learner confusion during task-oriented human-human tutorial dialogue. The model leverages textual dialogue, task, and facial expression history to predict upcoming confusion within a hidden Markov modeling framework. Analysis of the model structure also reveals meaningful modes of interaction within the tutoring sessions. The results demonstrate that because of its predictive power and rich qualitative representation, the model holds promise for informing the design of affective-sensitive tutoring systems.