An experimental comparison of the effectiveness of the all-uses and all-edges adequacy criteria
TAV4 Proceedings of the symposium on Testing, analysis, and verification
An experimental determination of sufficient mutant operators
ACM Transactions on Software Engineering and Methodology (TOSEM)
Is mutation an appropriate tool for testing experiments?
Proceedings of the 27th international conference on Software engineering
A Controlled Experiment Assessing Test Case Prioritization Techniques via Mutation Faults
ICSM '05 Proceedings of the 21st IEEE International Conference on Software Maintenance
An empirical framework for comparing effectiveness of testing and property-based formal analysis
PASTE '05 Proceedings of the 6th ACM SIGPLAN-SIGSOFT workshop on Program analysis for software tools and engineering
Automated, contract-based user testing of commercial-off-the-shelf components
Proceedings of the 28th international conference on Software engineering
MuJava: a mutation system for java
Proceedings of the 28th international conference on Software engineering
The class-level mutants of MuJava
Proceedings of the 2006 international workshop on Automation of software test
TimeAware test suite prioritization
Proceedings of the 2006 international symposium on Software testing and analysis
An empirical analysis and comparison of random testing techniques
Proceedings of the 2006 ACM/IEEE international symposium on Empirical software engineering
Using Mutation Analysis for Assessing and Comparing Testing Coverage Criteria
IEEE Transactions on Software Engineering
On the Use of Mutation Faults in Empirical Assessments of Test Case Prioritization Techniques
IEEE Transactions on Software Engineering
Sufficient mutation operators for measuring test effectiveness
Proceedings of the 30th international conference on Software engineering
Model-Based Tests for Access Control Policies
ICST '08 Proceedings of the 2008 International Conference on Software Testing, Verification, and Validation
Using a pilot study to derive a GUI model for automated testing
ACM Transactions on Software Engineering and Methodology (TOSEM)
Evaluating Test Suites and Adequacy Criteria Using Simulation-Based Models of Distributed Systems
IEEE Transactions on Software Engineering
The influence of size and coverage on test suite effectiveness
Proceedings of the eighteenth international symposium on Software testing and analysis
Automatic system testing of programs without test oracles
Proceedings of the eighteenth international symposium on Software testing and analysis
Time-aware test-case prioritization using integer linear programming
Proceedings of the eighteenth international symposium on Software testing and analysis
A family of code coverage-based heuristics for effective fault localization
Journal of Systems and Software
Is operator-based mutant selection superior to random mutant selection?
Proceedings of the 32nd ACM/IEEE International Conference on Software Engineering - Volume 1
Mutation-driven generation of unit tests and oracles
Proceedings of the 19th international symposium on Software testing and analysis
Augmenting automatically generated unit-test suites with regression oracle checking
ECOOP'06 Proceedings of the 20th European conference on Object-Oriented Programming
CarFast: achieving higher statement coverage faster
Proceedings of the ACM SIGSOFT 20th International Symposium on the Foundations of Software Engineering
Proceedings of the 2013 International Symposium on Software Testing and Analysis
Hi-index | 0.00 |
Mutation analysts has emerged as a standard approach for empirical assessment of testing techniques. The test practitioners decide about cost-effectiveness of testing strategies based on the number of mutants the testing techniques detect. Though fundamental rigor to empirical software testing, the use of mutants in the absence of real-world faults has raised the concern of whether mutants and real faults exhibit similar properties. This paper revisits this important concern and disseminates interesting findings regarding mutants and whether these synthetic faults can predict fault detection ability of test suites. The results of controlled experiments conducted in this paper show that mutation when used in testing experiments is highly sensitive to external threats caused by some influential factors including mutation operators, test suite size, and programming languages. This paper raises the awareness message of the use of mutation in testing experiment and suggests that any interpretation or generalization of experimental findings based on mutation should be justified according to the influential factors involved.