An evaluation of pedagogical tutorial tactics for a natural language tutoring system: a reinforcement learning approach

  • Authors:
  • Min Chi;Kurt VanLehn;Diane Litman;Pamela Jordan

  • Affiliations:
  • Human-Sciences and Technologies Advanced Research Institute, Stanford University, CA;School of Computing, Informatics and Decision Science Engineering, Arizona State University, AZ;Department of Computer Science and Intelligent Systems Program and Learning Research and Development Center, University of Pittsburgh, Pittsburgh, PA;Learning Research and Development Center, University of Pittsburgh, Pittsburgh, PA

  • Venue:
  • International Journal of Artificial Intelligence in Education - Special issue on Best of ITS 2010
  • Year:
  • 2011

Quantified Score

Hi-index 0.00

Visualization

Abstract

Pedagogical strategies are policies for a tutor to decide the next action when there are multiple actions available. When the content is controlled to be the same across experimental conditions, there has been little evidence that tutorial decisions have an impact on students' learning. In this paper, we applied Reinforcement Learning (RL) to induce two sets of pedagogical policies from pre-existing human interaction data. The NormGain set was derived with the goal of enhancing tutorial decisions that contribute to learning while the InvNormGain set was derived with the goal of enhancing those decisions that contribute less or even nothing to learning. The two sets were then tested with human students. Our results show that when the content was controlled to be the same, different pedagogical policies did make a difference in learning and more specifically, the NormGain students outperformed their peers. Overall our results suggest that content exposure and practice opportunities can help students to learn even when tutors have poor pedagogical tutorial tactics. However, with effective tutorial tactics, students can learn even more.