Comparative Assessment of Testing and Model Checking Using Program Mutation

Authors:
Jeremy S. Bradbury;James R. Cordy;Juergen Dingel
Affiliations:
University of Ontario Institute of Technology;Queen's University;Queen's University
Venue:
TAICPART-MUTATION '07 Proceedings of the Testing: Academic and Industrial Conference Practice and Research Techniques - MUTATION
Year:
2007

Citing 0
Cited 5

Higher Order Mutation Testing

Information and Software Technology
A method for detecting unusual defects in enterprise system using model checking techniques

SEPADS'11 Proceedings of the 10th WSEAS international conference on Software engineering, parallel and distributed systems
Comparing metaheuristic algorithms for error detection in java programs

SSBSE'11 Proceedings of the Third international conference on Search based software engineering
Using combinatorial benchmark construction to improve the assessment of concurrency bug detection tools

Proceedings of the 2012 Workshop on Parallel and Distributed Systems: Testing, Analysis, and Debugging
Selective mutation testing for concurrent code

Proceedings of the 2013 International Symposium on Software Testing and Analysis

Quantified Score

Hi-index	0.00

Visualization

Abstract

Developing correct concurrent code is more difficult than developing correct sequential code. This difficulty is due in part to the many different, possibly unexpected, executions of the program, and leads to the need for special quality assurance techniques for concurrent programs such as randomized testing and state space exploration. In this paper an approach is used that assesses testing and formal analysis tools using metrics to measure the effectiveness and efficiency of each technique at finding concurrency bugs. Using program mutation, the assessment method creates a range of faulty versions of a program and then evaluates the ability of various testing and formal analysis tools to detect these faults. The approach is implemented and automated in an experimental mutation analysis framework (ExMAn) which allows results to be more easily reproducible. To demonstrate the approach, we present the results of a comparison of testing using the IBM tool ConTest and model checking using the NASA tool Java PathFinder (JPF).