The use of program profiling for software maintenance with applications to the year 2000 problem
ESEC '97/FSE-5 Proceedings of the 6th European SOFTWARE ENGINEERING conference held jointly with the 5th ACM SIGSOFT international symposium on Foundations of software engineering
Visualization of test information to assist fault localization
Proceedings of the 24th International Conference on Software Engineering
Isolating cause-effect chains from computer programs
Proceedings of the 10th ACM SIGSOFT symposium on Foundations of software engineering
Pinpoint: Problem Determination in Large, Dynamic Internet Services
DSN '02 Proceedings of the 2002 International Conference on Dependable Systems and Networks
ICSM '01 Proceedings of the IEEE International Conference on Software Maintenance (ICSM'01)
Metamorphic Testing and Beyond
STEP '03 Proceedings of the Eleventh Annual International Workshop on Software Technology and Engineering Practice
Smart debugging software architectural design in SDL
Journal of Systems and Software - Special issue: Computer software & applications
Scalable statistical bug isolation
Proceedings of the 2005 ACM SIGPLAN conference on Programming language design and implementation
An effective testing method for end-user programmers
WEUSE I Proceedings of the first workshop on End-user software engineering
Empirical evaluation of the tarantula automatic fault-localization technique
Proceedings of the 20th IEEE/ACM international Conference on Automated software engineering
An empirical comparison between direct and indirect test result checking approaches
Proceedings of the 3rd international workshop on Software quality assurance
An Evaluation of Similarity Coefficients for Software Fault Localization
PRDC '06 Proceedings of the 12th Pacific Rim International Symposium on Dependable Computing
Proceedings of the 2007 international symposium on Software testing and analysis
On the Accuracy of Spectrum-based Fault Localization
TAICPART-MUTATION '07 Proceedings of the Testing: Academic and Industrial Conference Practice and Research Techniques - MUTATION
A Crosstab-based Statistical Method for Effective Fault Localization
ICST '08 Proceedings of the 2008 International Conference on Software Testing, Verification, and Validation
Bioinformatics
Adaptive Random Testing: The ART of test case diversity
Journal of Systems and Software
A family of code coverage-based heuristics for effective fault localization
Journal of Systems and Software
Application of Metamorphic Testing to Supervised Classifiers
QSIC '09 Proceedings of the 2009 Ninth International Conference on Quality Software
Semi-Proving: An Integrated Method for Program Proving, Testing, and Debugging
IEEE Transactions on Software Engineering
Testing and validating machine learning classifiers by metamorphic testing
Journal of Systems and Software
Spectrum-Based Fault Localization: Testing Oracles are No Longer Mandatory
QSIC '11 Proceedings of the 2011 11th International Conference on Quality Software
Hi-index | 0.00 |
Context: Because of its simplicity and effectiveness, Spectrum-Based Fault Localization (SBFL) has been one of the popular approaches towards fault localization. It utilizes the execution result of failure or pass, and the corresponding coverage information (such as program slice) to estimate the risk of being faulty for each program entity (such as statement). However, all existing SBFL techniques assume the existence of a test oracle to determine the execution result of a test case. But, it is common that test oracles do not exist, and hence the applicability of SBFL has been severely restricted. Objective: We aim at developing a framework that can extend the application of SBFL to the common situations where test oracles do not exist. Method: Our approach uses a new concept of metamorphic slice resulting from the integration of metamorphic testing and program slicing. In SBFL, instead of using the program slice and the result of failure or pass for an individual test case, a metamorphic slice and the result of violation or non-violation of a metamorphic relation are used. Since we need not know the execution result for an individual test case, the existence of a test oracle is no longer a requirement to apply SBFL. Results: An experimental study involving nine programs and three risk evaluation formulas was conducted. The results show that our proposed solution delivers a performance comparable to the performance observed by existing SBFL techniques for the situations where test oracles exist. Conclusion: With respect to the problem that SBFL is only applicable to programs with test oracles, we propose an innovative solution. Our solution is not only intuitively appealing and conceptually feasible, but also practically effective. Consequently, test oracles are no longer mandatory for SBFL, and hence the applicability of SBFL is significantly extended.