Defect Detection Efficiency: Test Case Based vs. Exploratory Testing

Authors:
Juha Itkonen;Mika V. Mantyla;Casper Lassenius
Affiliations:
Helsinki University of Technology, Finland;Helsinki University of Technology, Finland;Helsinki University of Technology, Finland
Venue:
ESEM '07 Proceedings of the First International Symposium on Empirical Software Engineering and Measurement
Year:
2007

Citing 0
Cited 3

How do testers do it? An exploratory study on manual testing practices

ESEM '09 Proceedings of the 2009 3rd International Symposium on Empirical Software Engineering and Measurement
More testers - The effect of crowd size and time restriction in software testing

Information and Software Technology
Are test cases needed? Replicated comparison between exploratory and test-case-based software testing

Empirical Software Engineering

Quantified Score

Hi-index	0.00

Visualization

Abstract

This paper presents a controlled experiment comparing the defect detection efficiency of exploratory testing (ET) and test case based testing (TCT). While traditional testing literature emphasizes test cases, ET stresses the individual tester's skills during test execution and does not rely upon predesigned test cases. In the experiment, 79 advanced software engineering students performed manual functional testing on an open-source application with actual and seeded defects. Each student participated in two 90-minute controlled sessions, using ET in one and TCT in the other. We found no significant differences in defect detection efficiency between TCT and ET. The distributions of detected defects did not differ significantly regarding technical type, detection difficulty, or severity. However, TCT produced significantly more false defect reports than ET. Surprisingly, our results show no benefit of using predesigned test cases in terms of defect detection efficiency, emphasizing the need for further studies of manual testing.