Automatic recognition of students' sorting algorithm implementations in a data structures and algorithms course

  • Authors:

  • Ahmad Taherkhani;Ari Korhonen;Lauri Malmi

  • Affiliations:

  • Aalto University, FI, Aalto, Finland;Aalto University, Aalto, Finland;Aalto University, Aalto, Finland

  • Venue:
  • Proceedings of the 12th Koli Calling International Conference on Computing Education Research
  • Year:
  • 2012

Quantified Score

Hi-index 0.00

Visualization

Abstract

Computing educators often rely on black-box analysis to assess students' work automatically and give feedback. This approach does not allow analyzing the quality of programs and checking if they implement the required algorithm. We introduce an instrument for recognizing and classifying algorithms (Aari) in terms of white-box testing to identify authentic students' sorting algorithm implementations in a data structures and algorithms course. Aari uses machine learning techniques to classify new instances. The students were asked to submit a program to sort an array of integers in two rounds: at the beginning of the course before sorting algorithms were introduced, and after taking a lecture on sorting algorithms. We evaluated the performance of Aari with the implementations of each round separately. The results show that the sorting algorithms, which Aari has been trained to recognize, are recognized with an average accuracy of about 90%. When considering all the submitted sorting algorithm implementations (including the variations of the standard algorithms), Aari achieved an overall accuracy of 71% and 81% for the first and second round, respectively. In addition, we analyzed the students' implementations manually to gain a better understanding of the reasons of failure in the recognition process. This analysis revealed that students have many misconceptions related to sorting algorithms, which results in problematic implementations that are more inefficient compared with those of standard algorithms. We discuss these variations along with the application of the tool in an educational context, its limitations and some directions for future work.