Empirical evaluation on FBD model-based test coverage criteria using mutation analysis

Authors:
Donghwan Shin;Eunkyoung Jee;Doo-Hwan Bae
Affiliations:
Dept. of Computer Science, KAIST, Daejeon, Republic of Korea;Dept. of Computer Science, KAIST, Daejeon, Republic of Korea;Dept. of Computer Science, KAIST, Daejeon, Republic of Korea
Venue:
MODELS'12 Proceedings of the 15th international conference on Model Driven Engineering Languages and Systems
Year:
2012

Citing 9
Cited 1

A Fortran language system for mutation-based software testing

Software—Practice & Experience
An experimental determination of sufficient mutant operators

ACM Transactions on Software Engineering and Methodology (TOSEM)
Experiments of the effectiveness of dataflow- and controlflow-based test adequacy criteria

ICSE '94 Proceedings of the 16th international conference on Software engineering
Structural test coverage criteria for lustre programs

Proceedings of the 10th international workshop on Formal methods for industrial critical systems
Using Mutation Analysis for Assessing and Comparing Testing Coverage Criteria

IEEE Transactions on Software Engineering
A data flow-based structural testing technique for FBD programs

Information and Software Technology
An Experimental Comparison of Four Unit Test Criteria: Mutation, Edge-Pair, All-Uses and Prime Path Coverage

ICSTW '09 Proceedings of the IEEE International Conference on Software Testing, Verification, and Validation Workshops
Generating tests from UML specifications

UML'99 Proceedings of the 2nd international conference on The unified modeling language: beyond the standard
An Analysis and Survey of the Development of Mutation Testing

IEEE Transactions on Software Engineering

Defining a test coverage criterion for model-level testing of FBD programs

Information and Software Technology

Quantified Score

Hi-index	0.00

Visualization

Abstract

Function Block Diagram (FBD), one of the PLC programming languages, is a graphical modeling language which has been increasingly used to implement safety-critical software such as nuclear reactor protection software. With increased importance of structural testing for FBD models, FBD model-based test coverage criteria have been introduced. In this paper, we empirically evaluate the fault detection effectiveness of the FBD coverage criteria using mutation analysis. We produce 1800 test suites satisfying the FBD criteria and generate more than 600 mutants automatically for the target industrial FBD models. Then we evaluate mutant detection of the test suites to assess the fault detection effectiveness of the coverage criteria. Based on the experimental results, we analyze strengths and weaknesses of the FBD coverage criteria, and suggest possible improvements for the test coverage criteria.