Modeling consumer-perceived web application fault severities for testing

  • Authors:

  • Kinga Dobolyi;Westley Weimer

  • Affiliations:

  • University of Virginia, Charlottesville, USA;University of Virginia, Charlottesville, USA

  • Venue:
  • Proceedings of the 19th international symposium on Software testing and analysis
  • Year:
  • 2010

Quantified Score

Hi-index 0.00

Visualization

Abstract

Despite the growing usage of web applications, extreme resource constraints during their development frequently leave them inadequately tested. Because testing may be perceived as having a low return on investment for web applications, we believe that providing a consumer-perceived fault severity model could allow developers to prioritize faults according to their likelihood of impacting consumer retention, encouraging web application developers to test more effectively. In a study involving 386 humans and 800 web application faults, we observe that an arbitrary human judgment of fault severity is unreliable. We thus present two models of fault severity that outperform individual humans in terms of correctly predicting the average consumer-perceived severity of web application faults. Our first model uses human annotations of fault surface features, and is 87% accurate at identifying low-priority, non-severe faults. We also present a fully automated conservative model that correctly identifies 55% of non-severe faults without missing any severe faults. Both models outperform humans at flagging severe faults, and can substitute or reinforce humans by prioritizing faults encountered in web application development and testing.