An Empirical Evaluation of Three Defect-Detection Techniques
Proceedings of the 5th European Software Engineering Conference
Reviewing 25 Years of Testing Technique Experiments
Empirical Software Engineering
A Critical Analysis of Empirical Research in Software Testing
ESEM '07 Proceedings of the First International Symposium on Empirical Software Engineering and Measurement
Guide to Advanced Empirical Software Engineering
Guide to Advanced Empirical Software Engineering
On the difficulty of replicating human subjects studies in software engineering
Proceedings of the 30th international conference on Software engineering
IEEE Software
The role of controlled experiments in software engineering research
Proceedings of the 2006 international conference on Empirical software engineering issues: critical assessment and future directions
ICST '12 Proceedings of the 2012 IEEE Fifth International Conference on Software Testing, Verification and Validation
| Hi-index | 0.00 |
Context: Many empirical studies have been carried out to evaluate software testing methods in the past decades. However, we are still not able to generalize the results, as most studies are not complete and differ significantly from one another. Objective: To contribute to the existing knowledge base of testing techniques by evaluating three software testing methods a) code reading by stepwise abstraction, b) functional testing using equivalence partitioning and boundary value analysis, and c) structural testing using 100% branch, multiple-condition, loop, and relational-operator coverage using a well-defined and standard schema. Method: A controlled experiment was carried out with eighteen subjects who applied three techniques to three C programs in a fractional factorial experimental design. Results: There is no difference in techniques in terms of failure observation and fault finding effectiveness and the effectiveness depends on the program rather than technique. In case of failure observation efficiency, all the techniques performed equally; however, in terms of fault isolation efficiency, code reading performed better than structural testing which in turn out performed functional testing. With respect to the fault types, all techniques performed equally except in case of cosmetic faults where functional testing performed better than other two testing methods. Conclusion: We conclude that all techniques are all equivalent in terms of effectiveness. However, the techniques differ partially in terms of efficiency. The effect of the program was significant in almost all cases. We need to build some standardized and better laboratory packages which should represent actual software engineering practices. Carry out experiments on such packages will help in deriving realistic results.