Selecting Software Test Data Using Data Flow Information
IEEE Transactions on Software Engineering
Software testing techniques (2nd ed.)
Software testing techniques (2nd ed.)
An experimental comparison of the effectiveness of the all-uses and all-edges adequacy criteria
TAV4 Proceedings of the symposium on Testing, analysis, and verification
Investigations of the software testing coupling effect
ACM Transactions on Software Engineering and Methodology (TOSEM)
An experimental determination of sufficient mutant operators
ACM Transactions on Software Engineering and Methodology (TOSEM)
Experiments of the effectiveness of dataflow- and controlflow-based test adequacy criteria
ICSE '94 Proceedings of the 16th international conference on Software engineering
Further empirical studies of test effectiveness
SIGSOFT '98/FSE-6 Proceedings of the 6th ACM SIGSOFT international symposium on Foundations of software engineering
Experimentation in software engineering: an introduction
Experimentation in software engineering: an introduction
Prioritizing Test Cases For Regression Testing
IEEE Transactions on Software Engineering
Software Metrics: A Rigorous and Practical Approach
Software Metrics: A Rigorous and Practical Approach
An Experimental Comparison of the Effectiveness of Branch Testing and Data Flow Testing
IEEE Transactions on Software Engineering
Improving test suites via operational abstraction
Proceedings of the 25th International Conference on Software Engineering
Empirical Evaluation of the Textual Differencing Regression Testing Technique
ICSM '98 Proceedings of the International Conference on Software Maintenance
General Test Result Checking with Log File Analysis
IEEE Transactions on Software Engineering
Using Simulation to Empirically Investigate Test Coverage Criteria Based on Statechart
Proceedings of the 26th International Conference on Software Engineering
Is mutation an appropriate tool for testing experiments?
Proceedings of the 27th international conference on Software engineering
Testing Programs with the Aid of a Compiler
IEEE Transactions on Software Engineering
ICSE COMPANION '07 Companion to the proceedings of the 29th International Conference on Software Engineering
Sufficient mutation operators for measuring test effectiveness
Proceedings of the 30th international conference on Software engineering
Configuration-aware regression testing: an empirical study of sampling and prioritization
ISSTA '08 Proceedings of the 2008 international symposium on Software testing and analysis
Inter-context control-flow and data-flow test adequacy criteria for nesC applications
Proceedings of the 16th ACM SIGSOFT International Symposium on Foundations of software engineering
On guiding the augmentation of an automated test suite via mutation analysis
Empirical Software Engineering
ICSE '09 Proceedings of the 31st International Conference on Software Engineering
Testing for trustworthiness in scientific software
SECSE '09 Proceedings of the 2009 ICSE Workshop on Software Engineering for Computational Science and Engineering
The influence of size and coverage on test suite effectiveness
Proceedings of the eighteenth international symposium on Software testing and analysis
Using machine learning to refine Category-Partition test specifications and test suites
Information and Software Technology
MODELS '09 Proceedings of the 12th International Conference on Model Driven Engineering Languages and Systems
Masking boundary value coverage: effectiveness and efficiency
TAIC PART'10 Proceedings of the 5th international academic and industrial conference on Testing - practice and research techniques
A generic approach to run mutation analysis
TAIC PART'10 Proceedings of the 5th international academic and industrial conference on Testing - practice and research techniques
Chronological fault-based mutation processes for WS-BPEL 2.0 programs
International Journal of Web Engineering and Technology
A tester-assisted methodology for test redundancy detection
Advances in Software Engineering - Special issue on software test automation
Evolutionary repair of faulty software
Applied Soft Computing
Diagnosing new faults using mutants and prior faults (NIER track)
Proceedings of the 33rd International Conference on Software Engineering
Adaptive random testing: an illusion of effectiveness?
Proceedings of the 2011 International Symposium on Software Testing and Analysis
The use of mutation in testing experiments and its sensitivity to external threats
Proceedings of the 2011 International Symposium on Software Testing and Analysis
Test adequacy criterion based on coincidental correctness probability
Proceedings of the Second Asia-Pacific Symposium on Internetware
Fault localization for data-centric programs
Proceedings of the 19th ACM SIGSOFT symposium and the 13th European conference on Foundations of software engineering
Information and Software Technology
Accounting for defect characteristics in evaluations of testing techniques
ACM Transactions on Software Engineering and Methodology (TOSEM)
On generating mutants for AspectJ programs
Information and Software Technology
Proceedings of the 2012 International Symposium on Software Testing and Analysis
Object-Oriented testing capabilities and performance evaluation of the c# mutation system
CEE-SET'09 Proceedings of the 4th IFIP TC 2 Central and East European conference on Advances in Software Engineering Techniques
Combining UML sequence and state machine diagrams for data-flow based integration testing
ECMFA'12 Proceedings of the 8th European conference on Modelling Foundations and Applications
Empirical evaluation on FBD model-based test coverage criteria using mutation analysis
MODELS'12 Proceedings of the 15th international conference on Model Driven Engineering Languages and Systems
SSBSE'12 Proceedings of the 4th international conference on Search Based Software Engineering
Comparing non-adequate test suites using coverage criteria
Proceedings of the 2013 International Symposium on Software Testing and Analysis
Observable modified Condition/Decision coverage
Proceedings of the 2013 International Conference on Software Engineering
Bridging the gap between the total and additional test-case prioritization strategies
Proceedings of the 2013 International Conference on Software Engineering
Using fault history to improve mutation reduction
Proceedings of the 2013 9th Joint Meeting on Foundations of Software Engineering
Microprocessors & Microsystems
Combining mutation and fault localization for automated program debugging
Journal of Systems and Software
An empirical study on the use of mutant traces for diagnosis of faults in deployed systems
Journal of Systems and Software
Hi-index | 0.00 |
The empirical assessment of test techniques plays an important role in software testing research. One common practice is to seed faults in subject software, either manually or by using a program that generates all possible mutants based on a set of mutation operators. The latter allows the systematic, repeatable seeding of large numbers of faults, thus facilitating the statistical analysis of fault detection effectiveness of test suites; however, we do not know whether empirical results obtained this way lead to valid, representative conclusions. Focusing on four common control and data flow criteria (Block, Decision, C-Use, and P-Use), this paper investigates this important issue based on a middle size industrial program with a comprehensive pool of test cases and known faults. Based on the data available thus far, the results are very consistent across the investigated criteria as they show that the use of mutation operators is yielding trustworthy results: Generated mutants can be used to predict the detection effectiveness of real faults. Applying such a mutation analysis, we then investigate the relative cost and effectiveness of the above-mentioned criteria by revisiting fundamental questions regarding the relationships between fault detection, test suite size, and control/data flow coverage. Although such questions have been partially investigated in previous studies, we can use a large number of mutants, which helps decrease the impact of random variation in our analysis and allows us to use a different analysis approach. Our results are then compared with published studies, plausible reasons for the differences are provided, and the research leads us to suggest a way to tune the mutation analysis process to possible differences in fault detection probabilities in a specific environment.