On establishing a benchmark for evaluating static analysis alert prioritization and classification techniques

  • Authors:

  • Sarah Heckman;Laurie Williams

  • Affiliations:

  • North Carolina State University, Raleigh, NC, USA;North Carolina State University, Raleigh, NC, USA

  • Venue:
  • Proceedings of the Second ACM-IEEE international symposium on Empirical software engineering and measurement
  • Year:
  • 2008

Quantified Score

Hi-index 0.00

Visualization

Abstract

Benchmarks provide an experimental basis for evaluating software engineering processes or techniques in an objective and repeatable manner. We present the FAULTBENCH v0.1 benchmark, as a contribution to current benchmark materials, for evaluation and comparison of techniques that prioritize and classify alerts generated by static analysis tools. Static analysis tools may generate an overwhelming number of alerts, the majority of which are likely to be false positives (FP). Two FP mitigation techniques, alert prioritization and classification, provide an ordering or classification of alerts, identifying those likely to be anomalies. We evaluate FAULTBENCH using three versions of a FP mitigation technique within the AWARE adaptive prioritization model. Individual FAULTBENCH subjects vary in their optimal FP mitigation techniques. Together, FAULTBENCH subjects provide a precise and general evaluation of FP mitigation techniques.