An empirical study on the effectiveness of security code review

  • Authors:

  • Anne Edmundson;Brian Holtkamp;Emanuel Rivera;Matthew Finifter;Adrian Mettler;David Wagner

  • Affiliations:

  • Cornell University, Ithaca, NY;University of Houston---Downtown, Houston, TX;Polytechnic University of Puerto Rico, San Juan, Puerto Rico;University of California, Berkeley, CA;University of California, Berkeley, CA;University of California, Berkeley, CA

  • Venue:
  • ESSoS'13 Proceedings of the 5th international conference on Engineering Secure Software and Systems
  • Year:
  • 2013

Quantified Score

Hi-index 0.00

Visualization

Abstract

With the rise of the web as a dominant application platform, web security vulnerabilities are of increasing concern. Ideally, the web application development process would detect and correct these vulnerabilities before they are released to the public. This research aims to quantify the effectiveness of software developers at security code review as well as determine the variation in effectiveness among web developers. We hired 30 developers to conduct a manual code review of a small web application. The web application supplied to developers had seven known vulnerabilities, including three different types: Cross-Site Scripting, Cross-Site Request Forgery, and SQL Injection. Our findings include: (1) none of the subjects found all confirmed vulnerabilities, (2) more experience does not necessarily mean that the reviewer will be more accurate or effective, and (3) reports of false vulnerabilities were significantly correlated with reports of valid vulnerabilities.