ICSE '07 Proceedings of the 29th international conference on Software Engineering
Automation of broad sanity test generation
Programming and Computing Software
On the danger of coverage directed test case generation
FASE'12 Proceedings of the 15th international conference on Fundamental Approaches to Software Engineering
Up and out: scaling formal analysis using model-based development and architecture modeling
Proceedings of the 2013 ACM SIGAda annual conference on High integrity language technology
| Hi-index | 0.00 |
When using tools to automatically generate tests-suites from a specification, the selection of coverage criterion that guides the generation process is of imperative importance. In a previous study that evaluated test generation with model checking, we observed that although a coverage criterion may seem reasonable when instrumenting a model or code to measure the adequacy of a test suite, it may be unsuitable when formalized and used to guide the model checker to generate a test suite; the generated tests technically provide adequate coverage according to the formalization, but do so in a way that exercises only small portions of the system under study and finds few faults. Based on those results, we concluded that fully automated test-suite generation techniques must be pursued with great caution and that coverage criteria specifically addressing test-suite generation from formal specifications are needed. In this report, we attempt to better understand these concerns by evaluating several coverage criteria that bring together aspects from condition and control based criteria. We evaluate the fault finding capability of the criteria on a close to production flight guidance system and discuss the opportunities and challenges that arise from the increased use of fully automated model-based testing.