Heuristic ranking of java program edits for fault localization

• Authors: Xiaoxia Ren;Barbara G. Ryder
• Venue: Proceedings of the 2007 international symposium on Software testing and analysis
• Year: 2007

In modern software development, regression tests are used to confirm the fundamental functionalities of an edited program and to assure the code quality. Difficulties occur when testing reveals unexpected behaviors, which indicate potential defects introduced by the edit. However, the changes that caused the failure(s) are not always easy to find. We propose a heuristic that ranks method changes that might have affected a failed test, indicating the likelihood that they may have contributed to a test failure. Our heuristic is based on the calling structure of the failed test (e.g., the number of ancestors and descendents of a method in the test's call graph, whether the caller or callee was changed, etc.). We evaluated the effectiveness of the heuristic in 14 pairs of edited versions in the Eclipse jdt core plug-in, using the test suite from its compiler tests plug-in. Our results indicate that when a failure is caused by a single method change, our heuristic ranked the failure-inducing change as number 1 or number 2 of all the method changes in 67% of the delegate tests (i.e., representatives of all failing tests). Even when the failure is caused by some combination of the changes, rather than a single change, our heuristic still helps.