Detecting gaming the system in constraint-based tutors

  • Authors:
  • Ryan S. J. D. Baker;Antonija Mitrović;Moffat Mathews

  • Affiliations:
  • Department of Social Science and Policy Studies, Worcester Polytechnic Institute, Worcester, MA;Intelligent Computer Tutoring Group, Computer Science and Software Engineering, University of Canterbury, New Zealand;Intelligent Computer Tutoring Group, Computer Science and Software Engineering, University of Canterbury, New Zealand

  • Venue:
  • UMAP'10 Proceedings of the 18th international conference on User Modeling, Adaptation, and Personalization
  • Year:
  • 2010

Quantified Score

Hi-index 0.00

Visualization

Abstract

Recently, detectors of gaming the system have been developed for several intelligent tutoring systems where the problem-solving process is reified, and gaming consists of systematic guessing and help abuse Constraint-based tutors differ from the tutors where gaming detectors have previously been developed on several dimensions: in particular, higher-level answers are assessed according to a larger number of finer-grained constraints, and feedback is split into levels rather than an entire help sequence being available at any time Correspondingly, help abuse behaviors differ, including behaviors such as rapidly repeating the same answer or blank answers to elicit answers We use text replay labeling in combination with educational data mining methods to create a gaming detector for SQL-Tutor, a popular constraint-based tutor This detector assesses gaming at the level of multiple-submission sequences and is accurate both at identifying gaming within submission sequences and at identifying how much each student games the system It achieves only limited success, however, at distinguishing different types of gaming behavior from each other.