The category-partition method for specifying and generating fuctional tests
Communications of the ACM
An experimental comparison of the effectiveness of the all-uses and all-edges adequacy criteria
TAV4 Proceedings of the symposium on Testing, analysis, and verification
Investigations of the software testing coupling effect
ACM Transactions on Software Engineering and Methodology (TOSEM)
An experimental determination of sufficient mutant operators
ACM Transactions on Software Engineering and Methodology (TOSEM)
Experiments of the effectiveness of dataflow- and controlflow-based test adequacy criteria
ICSE '94 Proceedings of the 16th international conference on Software engineering
Empirical Studies of a Safe Regression Test Selection Technique
IEEE Transactions on Software Engineering
Further empirical studies of test effectiveness
SIGSOFT '98/FSE-6 Proceedings of the 6th ACM SIGSOFT international symposium on Foundations of software engineering
Experimentation in software engineering: an introduction
Experimentation in software engineering: an introduction
An empirical study of regression test selection techniques
ACM Transactions on Software Engineering and Methodology (TOSEM)
Improving test suites via operational abstraction
Proceedings of the 25th International Conference on Software Engineering
Empirical Evaluation of the Textual Differencing Regression Testing Technique
ICSM '98 Proceedings of the International Conference on Software Maintenance
General Test Result Checking with Log File Analysis
IEEE Transactions on Software Engineering
Using Simulation to Empirically Investigate Test Coverage Criteria Based on Statechart
Proceedings of the 26th International Conference on Software Engineering
Empirical Software Engineering
Automated replay and failure detection for web applications
Proceedings of the 20th IEEE/ACM international Conference on Automated software engineering
An empirical framework for comparing effectiveness of testing and property-based formal analysis
PASTE '05 Proceedings of the 6th ACM SIGPLAN-SIGSOFT workshop on Program analysis for software tools and engineering
Prioritizing JUnit Test Cases: An Empirical Assessment and Cost-Benefits Analysis
Empirical Software Engineering
Automated, contract-based user testing of commercial-off-the-shelf components
Proceedings of the 28th international conference on Software engineering
MuJava: a mutation system for java
Proceedings of the 28th international conference on Software engineering
The class-level mutants of MuJava
Proceedings of the 2006 international workshop on Automation of software test
Random testing of formal software models and induced coverage
Proceedings of the 1st international workshop on Random testing
Tool support for randomized unit testing
Proceedings of the 1st international workshop on Random testing
TimeAware test suite prioritization
Proceedings of the 2006 international symposium on Software testing and analysis
An empirical analysis and comparison of random testing techniques
Proceedings of the 2006 ACM/IEEE international symposium on Empirical software engineering
Proceedings of the 14th ACM SIGSOFT international symposium on Foundations of software engineering
An empirical comparison between direct and indirect test result checking approaches
Proceedings of the 3rd international workshop on Software quality assurance
Adaptive random testing through iterative partitioning revisited
Proceedings of the 3rd international workshop on Software quality assurance
Information and Software Technology
Using Mutation Analysis for Assessing and Comparing Testing Coverage Criteria
IEEE Transactions on Software Engineering
On the Use of Mutation Faults in Empirical Assessments of Test Case Prioritization Techniques
IEEE Transactions on Software Engineering
ICSE COMPANION '07 Companion to the proceedings of the 29th International Conference on Software Engineering
Formal Software Analysis Emerging Trends in Software Model Checking
FOSE '07 2007 Future of Software Engineering
A practical approach to testing GUI systems
Empirical Software Engineering
Improving random test sets using the diversity oriented test data generation
Proceedings of the 2nd international workshop on Random testing: co-located with the 22nd IEEE/ACM International Conference on Automated Software Engineering (ASE 2007)
A state-based approach to integration testing based on UML models
Information and Software Technology
Fourth international workshop on Software quality assurance: in conjunction with the 6th ESEC/FSE joint meeting
Applying Concept Analysis to User-Session-Based Testing of Web Applications
IEEE Transactions on Software Engineering
Sufficient mutation operators for measuring test effectiveness
Proceedings of the 30th international conference on Software engineering
Towards a mutation-based automatic framework for evaluating code clone detection tools
Proceedings of the 2008 C3S2E conference
Towards Mutation Analysis for Lustre Programs
Electronic Notes in Theoretical Computer Science (ENTCS)
Using a pilot study to derive a GUI model for automated testing
ACM Transactions on Software Engineering and Methodology (TOSEM)
The Effectiveness of T-Way Test Data Generation
SAFECOMP '08 Proceedings of the 27th international conference on Computer Safety, Reliability, and Security
An empirical study of the effect of time constraints on the cost-benefits of regression testing
Proceedings of the 16th ACM SIGSOFT International Symposium on Foundations of software engineering
Requirements Coverage as an Adequacy Measure for Conformance Testing
ICFEM '08 Proceedings of the 10th International Conference on Formal Methods and Software Engineering
Journal of Systems and Software
A transformational language for mutant description
Computer Languages, Systems and Structures
System state coverage through automatic test case generation
International Journal of Information and Communication Technology
On guiding the augmentation of an automated test suite via mutation analysis
Empirical Software Engineering
Testing for trustworthiness in scientific software
SECSE '09 Proceedings of the 2009 ICSE Workshop on Software Engineering for Computational Science and Engineering
The influence of size and coverage on test suite effectiveness
Proceedings of the eighteenth international symposium on Software testing and analysis
Efficient mutation testing by checking invariant violations
Proceedings of the eighteenth international symposium on Software testing and analysis
Automatic system testing of programs without test oracles
Proceedings of the eighteenth international symposium on Software testing and analysis
Time-aware test-case prioritization using integer linear programming
Proceedings of the eighteenth international symposium on Software testing and analysis
Combining Genetic Algorithms and Mutation Testing to Generate Test Sequences
IWANN '09 Proceedings of the 10th International Work-Conference on Artificial Neural Networks: Part I: Bio-Inspired Systems: Computational and Ambient Intelligence
Conformance Testing of Network Simulators Based on Metamorphic Testing Technique
FMOODS '09/FORTE '09 Proceedings of the Joint 11th IFIP WG 6.1 International Conference FMOODS '09 and 29th IFIP WG 6.1 International Conference FORTE '09 on Formal Techniques for Distributed Systems
Should software testers use mutation analysis to augment a test set?
Journal of Systems and Software
A family of code coverage-based heuristics for effective fault localization
Journal of Systems and Software
MODELS '09 Proceedings of the 12th International Conference on Model Driven Engineering Languages and Systems
Is operator-based mutant selection superior to random mutant selection?
Proceedings of the 32nd ACM/IEEE International Conference on Software Engineering - Volume 1
Automating software tests with partial oracles in integrated environments
Proceedings of the 5th Workshop on Automation of Software Test
Towards automating the generation of mutation tests
Proceedings of the 5th Workshop on Automation of Software Test
Mutation-driven generation of unit tests and oracles
Proceedings of the 19th international symposium on Software testing and analysis
The influence of multiple artifacts on the effectiveness of software testing
Proceedings of the IEEE/ACM international conference on Automated software engineering
Scalable distributed concolic testing: a case study on a flash storage platform
ICTAC'10 Proceedings of the 7th International colloquium conference on Theoretical aspects of computing
A generic approach to run mutation analysis
TAIC PART'10 Proceedings of the 5th international academic and industrial conference on Testing - practice and research techniques
Issues in testing collection class libraries
Proceedings of the 1st Workshop on Testing Object-Oriented Systems
Contract-driven testing of javascript code
TOOLS'10 Proceedings of the 48th international conference on Objects, models, components, patterns
Proof process evaluation with mutation analysis
TAP'10 Proceedings of the 4th international conference on Tests and proofs
Testing techniques in software engineering
Testing techniques in software engineering
Chronological fault-based mutation processes for WS-BPEL 2.0 programs
International Journal of Web Engineering and Technology
Testing and validating machine learning classifiers by metamorphic testing
Journal of Systems and Software
On the estimation of adequate test set size using fault failure rates
Journal of Systems and Software
A tester-assisted methodology for test redundancy detection
Advances in Software Engineering - Special issue on software test automation
An empirical evaluation of P system testing techniques
Natural Computing: an international journal
Security mutation testing of the FileZilla FTP server
Proceedings of the 2011 ACM Symposium on Applied Computing
Using conditional mutation to increase the efficiency of mutation analysis
Proceedings of the 6th International Workshop on Automation of Software Test
On effective testing of health care simulation software
Proceedings of the 3rd Workshop on Software Engineering in Health Care
Is my model right? Let me ask the expert
Journal of Systems and Software
The use of mutation in testing experiments and its sensitivity to external threats
Proceedings of the 2011 International Symposium on Software Testing and Analysis
Concurrency-oriented verification and coverage of system-level designs
ACM Transactions on Design Automation of Electronic Systems (TODAES)
Automating unit and integration testing with partial oracles
Software Quality Control
SimTester: a controllable and observable testing framework for embedded systems
VEE '12 Proceedings of the 8th ACM SIGPLAN/SIGOPS conference on Virtual Execution Environments
Augmenting automatically generated unit-test suites with regression oracle checking
ECOOP'06 Proceedings of the 20th European conference on Object-Oriented Programming
TACAS'06 Proceedings of the 12th international conference on Tools and Algorithms for the Construction and Analysis of Systems
MAJOR: An efficient and extensible tool for mutation analysis in a Java compiler
ASE '11 Proceedings of the 2011 26th IEEE/ACM International Conference on Automated Software Engineering
Prioritizing tests for fault localization through ambiguity group reduction
ASE '11 Proceedings of the 2011 26th IEEE/ACM International Conference on Automated Software Engineering
On generating mutants for AspectJ programs
Information and Software Technology
On the danger of coverage directed test case generation
FASE'12 Proceedings of the 15th international conference on Fundamental Approaches to Software Engineering
Proceedings of the 2012 International Symposium on Software Testing and Analysis
Proceedings of the 2012 International Symposium on Software Testing and Analysis
On-demand test suite reduction
Proceedings of the 34th International Conference on Software Engineering
Proceedings of the 34th International Conference on Software Engineering
A taxonomy of model-based testing approaches
Software Testing, Verification & Reliability
Augmented dynamic symbolic execution
Proceedings of the 27th IEEE/ACM International Conference on Automated Software Engineering
A comparison of database fault detection capabilities using mutation testing
Proceedings of the ACM-IEEE international symposium on Empirical software engineering and measurement
Generation of TLM testbenches using mutation testing
Proceedings of the eighth IEEE/ACM/IFIP international conference on Hardware/software codesign and system synthesis
Component testing is not enough: a study of software faults in telecom middleware
TestCom'07/FATES'07 Proceedings of the 19th IFIP TC6/WG6.1 international conference, and 7th international conference on Testing of Software and Communicating Systems
Code-based prioritization: a pre-testing effort to minimize post-release failures
Innovations in Systems and Software Engineering
ConSMutate: SQL mutants for guiding concolic testing of database applications
ICFEM'12 Proceedings of the 14th international conference on Formal Engineering Methods: formal methods and software engineering
Science of Computer Programming
Mutation at the multi-class and system levels
Science of Computer Programming
A Metamorphic Relation-Based Approach to Testing Web Services Without Oracles
International Journal of Web Services Research
Evaluating test suite characteristics, cost, and effectiveness of FSM-based testing methods
Information and Software Technology
Test case generation from natural language requirements based on SCR specifications
Proceedings of the 28th Annual ACM Symposium on Applied Computing
Mutation testing strategies using mutant classification
Proceedings of the 28th Annual ACM Symposium on Applied Computing
Proceedings of the 2013 International Symposium on Software Testing and Analysis
Comparing non-adequate test suites using coverage criteria
Proceedings of the 2013 International Symposium on Software Testing and Analysis
Faster mutation testing inspired by test prioritization and reduction
Proceedings of the 2013 International Symposium on Software Testing and Analysis
Comparing multi-point stride coverage and dataflow coverage
Proceedings of the 2013 International Conference on Software Engineering
Bridging the gap between the total and additional test-case prioritization strategies
Proceedings of the 2013 International Conference on Software Engineering
Empirical answers to fundamental software engineering problems (panel)
Proceedings of the 2013 9th Joint Meeting on Foundations of Software Engineering
BPELDebugger: An effective BPEL-specific fault localization framework
Information and Software Technology
Diversity oriented test data generation using metaheuristic search techniques
Information Sciences: an International Journal
GUI testing assisted by human knowledge: Random vs. functional
Journal of Systems and Software
Combining mutation and fault localization for automated program debugging
Journal of Systems and Software
An empirical study on the use of mutant traces for diagnosis of faults in deployed systems
Journal of Systems and Software
Hi-index | 0.00 |
The empirical assessment of test techniques plays an important role in software testing research. One common practice is to instrument faults, either manually or by using mutation operators. The latter allows the systematic, repeatable seeding of large numbers of faults; however, we do not know whether empirical results obtained this way lead to valid, representative conclusions. This paper investigates this important question based on a number of programs with comprehensive pools of test cases and known faults. It is concluded that, based on the data available thus far, the use of mutation operators is yielding trustworthy results (generated mutants are similar to real faults). Mutants appear however to be different from hand-seeded faults that seem to be harder to detect than real faults.