Automatic Multi-criteria Assessment of Open-Ended Questions: A Case Study in School Algebra

  • Authors:

  • Élisabeth Delozanne;Dominique Prévit;Brigitte Grugeon;Françoise Chenevotot

  • Affiliations:

  • L'UTES, Université Paris VI, PARIS, France 75005;LIUM, Le Mans Cedex 9, France 72085;DIDIREM, Université Paris VII, PARIS Cedex 5, France 75251;DIDIREM, Université Paris VII, PARIS Cedex 5, France 75251

  • Venue:
  • ITS '08 Proceedings of the 9th international conference on Intelligent Tutoring Systems
  • Year:
  • 2008

Quantified Score

Hi-index 0.00

Visualization

Abstract

This paper deals with authoring assessments of complex competence involving open-ended questions. We present, PépiGen, a multi-criteria automatic assessor for school algebra, via a walkthrough of an example. PépiGen is based on our previous work on Pépite, an automatic cognitive diagnosis tool that capitalizes on educational research results. From that prototype, we derived patterns of diagnosis tasks. A pattern models (i) a class of exercises, (ii) the different students' points of view on the solutions reported in the literature or observed in a corpus, (iii) and a multidimensional assessment for each solution approach. To adapt an assessment to a specific classroom context (e.g. level of difficulty, time, learning objectives) an interface allows an IT non expert (e.g. a teacher) to generate new instances of exercises by filling the pattern parameters. The originality of our research lies in the fact that our system generates the automatic analysis of students' simple or complex answers, such as algebraic reasoning. This is an ongoing work but preliminary evaluation shows that PépiGen is already successful in generating and analyzing most answers on several classes of problems.