Hairball: lint-inspired static analysis of scratch projects

  • Authors:

  • Bryce Boe;Charlotte Hill;Michelle Len;Greg Dreschler;Phillip Conrad;Diana Franklin

  • Affiliations:

  • UC Santa Barbara, Santa Barbara, CA, USA;UC Santa Barbara, Santa Barbara, CA, USA;UC Santa Barbara, Santa Barbara, CA, USA;UC Santa Barbara, Santa Barbara, CA, USA;UC Santa Barbara, Santa Barbara, CA, USA;UC Santa Barbara, Santa Barbara, CA, USA

  • Venue:
  • Proceeding of the 44th ACM technical symposium on Computer science education
  • Year:
  • 2013

Quantified Score

Hi-index 0.00

Visualization

Abstract

Scratch programming has risen in prominence, not only as a potential language for K-12 computer science, but also in introductory college courses. Unfortunately, grading Scratch programs is time-consuming, requiring manual execution of each program. Automation of this process is greatly complicated by the very reason Scratch is an attractive introductory language--the projects are multimedia in nature, requiring eyes and ears to fully appreciate. We propose Hairball, an automated system that can be used both by a student to point out potential errors or unsafe practices, and by a grader to assist in inspecting the implementation of Scratch programs. Because automatic analysis will not be able to determine the sensory effect, Hairball focuses instead on the implementation, including safe/robust programming practices, providing a "lint-like" tool for Scratch. In this case study, we have created an initial set of Hairball plugins that detect and label instances of initialization of Scratch state, synchronization between say and sound blocks, synchronization between broadcast and receive blocks, and use of timing and loops for complex animation. Our evaluation shows that Hairball is very useful in conjunction with manual analysis. Overall, Hairball was actually slightly more accurate than manual analysis at labeling these instances. Specifically for broadcast/receive, Hairball's analysis correctly classified 99% of the 432 instances, manual analysis only 81%. Overall, if Hairball was only used to identify correctly implemented instances, with manual analysis for the remainder, it would remove 76% of the instances for the manual analysis and assist in the rest, with a false positive rate of less than 0.5%.