Reducing the cost of mutation testing: an empirical study
Journal of Systems and Software
Software metrics (2nd ed.): a rigorous and practical approach
Software metrics (2nd ed.): a rigorous and practical approach
Software fault injection: inoculating programs against errors
Software fault injection: inoculating programs against errors
The relationship between program dependence and mutation analysis
Mutation testing for the new century
Is mutation an appropriate tool for testing experiments?
Proceedings of the 27th international conference on Software engineering
MuJava: an automated class mutation system: Research Articles
Software Testing, Verification & Reliability
Testing aspect-oriented programming Pointcut Descriptors
Proceedings of the 2nd workshop on Testing aspect-oriented programs
A Candidate Fault Model for AspectJ Pointcuts
ISSRE '06 Proceedings of the 17th International Symposium on Software Reliability Engineering
Using Mutation Analysis for Assessing and Comparing Testing Coverage Criteria
IEEE Transactions on Software Engineering
Feedback-Directed Random Test Generation
ICSE '07 Proceedings of the 29th international conference on Software Engineering
On Identifying Bug Patterns in Aspect-Oriented Programs
COMPSAC '07 Proceedings of the 31st Annual International Computer Software and Applications Conference - Volume 01
Sufficient mutation operators for measuring test effectiveness
Proceedings of the 30th international conference on Software engineering
Mutation Testing for Aspect-Oriented Programs
ICST '08 Proceedings of the 2008 International Conference on Software Testing, Verification, and Validation
AjMutator: A Tool for the Mutation Analysis of AspectJ Pointcut Descriptors
ICSTW '09 Proceedings of the IEEE International Conference on Software Testing, Verification, and Validation Workshops
Automating the mutation testing of aspect-oriented Java programs
Proceedings of the 5th Workshop on Automation of Software Test
(Un-)Covering Equivalent Mutants
ICST '10 Proceedings of the 2010 Third International Conference on Software Testing, Verification and Validation
A Dataflow Testing Approach for Aspect-Oriented Programs
HASE '10 Proceedings of the 2010 IEEE 12th International Symposium on High-Assurance Systems Engineering
A Methodology for Collecting Valid Software Engineering Data
IEEE Transactions on Software Engineering
| Hi-index | 0.00 |
Context: Mutation analysis has been widely used in research studies to evaluate the effectiveness of test suites and testing techniques. Faulty versions (i.e., mutants) of a program are generated such that each mutant contains one seeded fault. The mutation score provides a measure of effectiveness. Objective: We study three problems with the use of mutation analysis for testing AspectJ programs:*The manual identification and removal of equivalent mutants is difficult and time consuming. We calculate the percentage of equivalent mutants generated for benchmark AspectJ programs using available mutation tools. *The generated mutants need to cover the various fault types described in the literature on fault models for AspectJ programs. We measure the distribution of the mutants generated using available mutation tools with respect to the AspectJ fault types. *We measure the difficulty of killing the generated mutants. We propose the use of simple analysis of the subject programs to prevent the generation of some equivalent mutants. Method: We revised existing AspectJ fault models and presented a fault model that removes the problems in existing fault models, such as overlapping between fault types and missing fault types. We also defined three new fault types that occur due to incorrect data-flow interactions occurring in AspectJ programs. We used three mutation tools: AjMutator, Proteum/AJ, and MuJava on three AspectJ programs. To measure the difficulty of killing the mutants created using a mutation operator, we compared the average number of the mutants killed by 10 test suites that satisfy block coverage criterion. Results: A high percentage of the mutants are equivalent. The mutation tools do not cover all the fault types. Only 4 out of 27 operators generated mutants that were easy to kill. Conclusions: Our analysis approach removed about 80% of the equivalent mutants. Higher order mutation is needed to cover all the fault types.