Improving student performance by evaluating how well students test their own programs
Journal on Educational Resources in Computing (JERIC)
Running students' software tests against each others' code: new life for an old "gimmick"
Proceedings of the 43rd ACM technical symposium on Computer Science Education
| Hi-index | 0.00 |
A normal industry standard measure, bug density (bugs per thousand non-commented source line of code), is a through mechanism to assess code quality. If it is used for evaluating students' code, students will realize their ability to write bug free code from professional context. The main issues of using bug density for object oriented languages are creating a comprehensive test suit, and running them against all solutions as the test cases are written as part of solutions may fail to compile against other codes. We provide a novel four phase Java specific solution: 1) developing a comprehensive master test suit by collecting all the students written valid test cases; 2) transforming the test cases to use late binding so that they can run against any solution; 3) running the entire tests against all the programs and removing redundant test suits; and 4) estimating bugs/KSLOC by determining the relationship between test case failures in the master suite and latent bugs hidden in student programs. The first two phases of this ongoing research are applied to two programming assignments in two different courses encompassing 147 student programs and 240,158 individual test cases. Experimental results show that we have indeed removed compile-time dependencies from test cases using late binding and thus, have resolved the main technical challenge of using bug density for accessing students' code. Our experimental results will help students to realize the quality of their code in terms of industry standard.