Prioritizing Test Cases For Regression Testing
IEEE Transactions on Software Engineering
Test Case Prioritization: A Family of Empirical Studies
IEEE Transactions on Software Engineering
Effectively prioritizing tests in development environment
ISSTA '02 Proceedings of the 2002 ACM SIGSOFT international symposium on Software testing and analysis
Visualization of test information to assist fault localization
Proceedings of the 24th International Conference on Software Engineering
A Study of Effective Regression Testing in Practice
ISSRE '97 Proceedings of the Eighth International Symposium on Software Reliability Engineering
Selecting a Cost-Effective Test Case Prioritization Technique
Software Quality Control
Locating causes of program failures
Proceedings of the 27th international conference on Software engineering
Proceedings of the 27th international conference on Software engineering
Scalable statistical bug isolation
Proceedings of the 2005 ACM SIGPLAN conference on Programming language design and implementation
SOBER: statistical model-based bug localization
Proceedings of the 10th European software engineering conference held jointly with 13th ACM SIGSOFT international symposium on Foundations of software engineering
Empirical Software Engineering
Improving test suites for efficient fault localization
Proceedings of the 28th international conference on Software engineering
A comparison of MC/DC, MUMCUT and several other coverage criteria for logical decisions
Journal of Systems and Software - Special issue: Quality software
Search Algorithms for Regression Test Case Prioritization
IEEE Transactions on Software Engineering
On the Accuracy of Spectrum-based Fault Localization
TAICPART-MUTATION '07 Proceedings of the Testing: Academic and Industrial Conference Practice and Research Techniques - MUTATION
An empirical study of the effects of test-suite reduction on fault localization
Proceedings of the 30th international conference on Software engineering
Fault localization using value replacement
ISSTA '08 Proceedings of the 2008 international symposium on Software testing and analysis
Capturing propagation of infected program states
Proceedings of the the 7th joint meeting of the European software engineering conference and the ACM SIGSOFT symposium on The foundations of software engineering
How Well Do Test Case Prioritization Techniques Support Statistical Fault Localization
COMPSAC '09 Proceedings of the 2009 33rd Annual IEEE International Computer Software and Applications Conference - Volume 01
Fault localization through evaluation sequences
Journal of Systems and Software
A family of code coverage-based heuristics for effective fault localization
Journal of Systems and Software
Adaptive Random Test Case Prioritization
ASE '09 Proceedings of the 2009 IEEE/ACM International Conference on Automated Software Engineering
Prioritizing Tests for Software Fault Localization
QSIC '10 Proceedings of the 2010 10th International Conference on Quality Software
On the Integration of Test Adequacy, Test Case Prioritization, and Statistical Fault Localization
QSIC '10 Proceedings of the 2010 10th International Conference on Quality Software
KLEE: unassisted and automatic generation of high-coverage tests for complex systems programs
OSDI'08 Proceedings of the 8th USENIX conference on Operating systems design and implementation
On Practical Adequate Test Suites for Integrated Test Case Prioritization and Fault Localization
QSIC '11 Proceedings of the 2011 11th International Conference on Quality Software
How well does test case prioritization integrate with statistical fault localization?
Information and Software Technology
Hi-index | 0.00 |
Context: Testing and debugging consume a significant portion of software development effort. Both processes are usually conducted independently despite their close relationship with each other. Test adequacy is vital for developers to assure that sufficient testing effort has been made, while finding all the faults in a program as soon as possible is equally important. A tight integration between testing and debugging activities is essential. Objective: The paper aims at finding whether three factors, namely, the adequacy criterion to gauge a test suite, the size of a prioritized test suite, and the percentage of such a test suite used in fault localization, have significant impacts on integrating test case prioritization techniques with statistical fault localization techniques. Method: We conduct a controlled experiment to investigate the effectiveness of applying adequate test suites to locate faults in a benchmark suite of seven Siemens programs and four real-life UNIX utility programs using three adequacy criteria, 16 test case prioritization techniques, and four statistical fault localization techniques. We measure the proportion of code needed to be examined in order to locate a fault as the effectiveness of statistical fault localization techniques. We also investigate the integration of test case prioritization and statistical fault localization with postmortem analysis. Result: The main result shows that on average, it is more effective for a statistical fault localization technique to utilize the execution results of a MC/DC-adequate test suite than those of a branch-adequate test suite, and is in turn more effective to utilize the execution results of a branch-adequate test suite than those of a statement-adequate test suite. On the other hand, we find that none of the fault localization techniques studied can be sufficiently effective in suggesting fault-relevant statements that can fit easily into one debug window of a typical IDE. Conclusion: We find that the adequacy criterion and the percentage of a prioritized test suite utilized are major factors affecting the effectiveness of statistical fault localization techniques. In our experiment, the adoption of a stronger adequacy criterion can lead to more effective integration of testing and debugging.