Orthogonal Latin squares: an application of experiment design to compiler testing
Communications of the ACM
Software error analysis: a real case study involving real faults and mutations
ISSTA '96 Proceedings of the 1996 ACM SIGSOFT international symposium on Software testing and analysis
An experimental determination of sufficient mutant operators
ACM Transactions on Software Engineering and Methodology (TOSEM)
Applying design of experiments to software testing: experience report
ICSE '97 Proceedings of the 19th international conference on Software engineering
All-uses vs mutation testing: an experimental comparison of effectiveness
Journal of Systems and Software
The AETG System: An Approach to Testing Based on Combinatorial Design
IEEE Transactions on Software Engineering
ADTEST: A Test Data Generation Suite for Ada Software Systems
IEEE Transactions on Software Engineering
Model-based testing in practice
Proceedings of the 21st international conference on Software engineering
Generating Software Test Data by Evolution
IEEE Transactions on Software Engineering
Instrumenting Programs With Flag Variables For Test Data Search By Genetic Algorithms
GECCO '02 Proceedings of the Genetic and Evolutionary Computation Conference
A Practical System for Mutation Testing: Help for the Common Programmer
Proceedings of the IEEE International Test Conference on TEST: The Next 25 Years
An Investigation of the Applicability of Design of Experiments to Software Testing
SEW '02 Proceedings of the 27th Annual NASA Goddard Software Engineering Workshop (SEW-27'02)
Model-based Testing of a Highly Programmable System
ISSRE '98 Proceedings of the The Ninth International Symposium on Software Reliability Engineering
Exploiting Symmetries to Test Programs
ISSRE '03 Proceedings of the 14th International Symposium on Software Reliability Engineering
A Practitioner's Guide to Software Test Design
A Practitioner's Guide to Software Test Design
Software Fault Interactions and Implications for Software Testing
IEEE Transactions on Software Engineering
Comparing the Fault Detection Effectiveness of N-way and Random Test Suites
ISESE '04 Proceedings of the 2004 International Symposium on Empirical Software Engineering
Is mutation an appropriate tool for testing experiments?
Proceedings of the 27th international conference on Software engineering
Search-based mutation testing for Simulink models
GECCO '05 Proceedings of the 7th annual conference on Genetic and evolutionary computation
Empirical Software Engineering
Generating good pseudo-random numbers
Computational Statistics & Data Analysis
IPOG: A General Strategy for T-Way Software Testing
ECBS '07 Proceedings of the 14th Annual IEEE International Conference and Workshops on the Engineering of Computer-Based Systems
The Csaw C Mutation Tool: Initial Results
TAICPART-MUTATION '07 Proceedings of the Testing: Academic and Industrial Conference Practice and Research Techniques - MUTATION
Experience with Path Analysis and Testing of Programs
IEEE Transactions on Software Engineering
Testing Programs with the Aid of a Compiler
IEEE Transactions on Software Engineering
| Hi-index | 0.00 |
This paper reports the results of a study comparing the effectiveness of automatically generated tests constructed using random and t-way combinatorial techniques on safety related industrial code using mutation adequacy criteria. A reference point is provided by hand generated test vectors constructed during development to establish minimum acceptance criteria. The study shows that 2-way testing is not adequate measured by mutants kill rate compared with hand generated test set of similar size, but that higher factor t-way test sets can perform at least as well. To reduce the computation overhead of testing large numbers of vectors over large numbers of mutants a staged optimising approach to applying t-way tests is proposed and evaluated which shows improvements in execution time and final test set size.