Checking Inside the Black Box: Regression Testing Based on Value Spectra Differences

Authors:
Tao Xie;David Notkin
Affiliations:
University of Washington;University of Washington
Venue:
ICSM '04 Proceedings of the 20th IEEE International Conference on Software Maintenance
Year:
2004

Citing 0
Cited 6

Checking Inside the Black Box: Regression Testing by Comparing Value Spectra

IEEE Transactions on Software Engineering
Staged deployment in mirage, an integrated software upgrade testing and distribution system

Proceedings of twenty-first ACM SIGOPS symposium on Operating systems principles
The causes of bloat, the limits of health

Proceedings of the 22nd annual ACM SIGPLAN conference on Object-oriented programming systems and applications
The Daikon system for dynamic detection of likely invariants

Science of Computer Programming
I-RMI: performance isolation in information flow applications

Proceedings of the ACM/IFIP/USENIX 2005 International Conference on Middleware
I-RMI: performance isolation in information flow applications

Middleware'05 Proceedings of the ACM/IFIP/USENIX 6th international conference on Middleware

Quantified Score

Hi-index	0.00

Visualization

Abstract

Comparing behaviors of program versions has become an important task in software maintenance and regression testing. Traditional regression testing strongly focuses on black-box comparison of program outputs. Program spectra have recently been proposed to characterize a programýs behavior inside the black box. Comparing program spectra of program versions offers insights into the internal behavior differences between versions. In this paper, we present a new class of program spectra, value spectra, which enriches the existing program spectra family. We compare the value spectra of an old version and a new version to detect internal behavior deviations in the new version. We use a deviation-propagation call tree to present the deviation details. Based on the deviation-propagation call tree, we propose two heuristics to locate deviation roots, which are program locations that trigger the behavior deviations. We have conducted an experiment on seven C programs to evaluate our approach. The results show that our approach can effectively expose program behavior differences between versions even when their program outputs are the same, and our approach reports deviation roots with high accuracy for most programs.