An Experimental Comparison of Four Unit Test Criteria: Mutation, Edge-Pair, All-Uses and Prime Path Coverage

Authors:
Nan Li;Upsorn Praphamontripong;Jeff Offutt
Affiliations:
-;-;-
Venue:
ICSTW '09 Proceedings of the IEEE International Conference on Software Testing, Verification, and Validation Workshops
Year:
2009

Citing 0
Cited 9

Better predicate testing

Proceedings of the 6th International Workshop on Automation of Software Test
Concurrency-oriented verification and coverage of system-level designs

ACM Transactions on Design Automation of Electronic Systems (TODAES)
An aspect-oriented reference architecture for Software Engineering Environments

Journal of Systems and Software
Automatically performing weak mutation with the aid of symbolic execution, concolic testing and search-based testing

Software Quality Control
A service-oriented reference architecture for software testing tools

ECSA'11 Proceedings of the 5th European conference on Software architecture
Mutation based test case generation via a path selection strategy

Information and Software Technology
Generation of TLM testbenches using mutation testing

Proceedings of the eighth IEEE/ACM/IFIP international conference on Hardware/software codesign and system synthesis
Empirical evaluation on FBD model-based test coverage criteria using mutation analysis

MODELS'12 Proceedings of the 15th international conference on Model Driven Engineering Languages and Systems
Improving logic-based testing

Journal of Systems and Software

Quantified Score

Hi-index	0.00

Visualization

Abstract

With recent increased expectations for quality, and the growth of agile processes and test driven development, developers are expected to do more and more effective unittesting. Yet, our knowledge of when to use the variousunit level test criteria is incomplete. The paper presentsresults from a comparison of four unit level software testing criteria. Mutation testing, prime path coverage, edgepair coverage, and all-uses testing were compared on twobases: the number of seeded faults found and the numberof tests needed to satisfy the criteria. The comparisonused a collection of Java classes taken from varioussources and hand-seeded faults. Tests were designed andgenerated mostly by hand with help from tools that computetest requirements and muJava. The findings are that mutationtests detected more faults and the other three criteriawere very similar. The paper also presents a secondarymeasure, a cost benefit ratio, computed as the number oftests needed to detect each fault. Surprisingly, mutation required the fewest number of tests. The paper also discusses some specific faults that were not found and presents analysis for why not.