Grouping target paths for evolutionary generation of test data in parallel
Journal of Systems and Software
Test suite generation with memetic algorithms
Proceedings of the 15th annual conference on Genetic and evolutionary computation
Does automated white-box test generation really help software testers?
Proceedings of the 2013 International Symposium on Software Testing and Analysis
Orthogonal exploration of the search space in evolutionary test case generation
Proceedings of the 2013 International Symposium on Software Testing and Analysis
Continuous test suite augmentation in software product lines
Proceedings of the 17th International Software Product Line Conference
Hi-index | 0.00 |
Not all bugs lead to program crashes, and not always is there a formal specification to check the correctness of a software test's outcome. A common scenario in software testing is therefore that test data are generated, and a tester manually adds test oracles. As this is a difficult task, it is important to produce small yet representative test sets, and this representativeness is typically measured using code coverage. There is, however, a fundamental problem with the common approach of targeting one coverage goal at a time: Coverage goals are not independent, not equally difficult, and sometimes infeasible—the result of test generation is therefore dependent on the order of coverage goals and how many of them are feasible. To overcome this problem, we propose a novel paradigm in which whole test suites are evolved with the aim of covering all coverage goals at the same time while keeping the total size as small as possible. This approach has several advantages, as for example, its effectiveness is not affected by the number of infeasible targets in the code. We have implemented this novel approach in the EvoSuite tool, and compared it to the common approach of addressing one goal at a time. Evaluated on open source libraries and an industrial case study for a total of 1,741 classes, we show that EvoSuite achieved up to 188 times the branch coverage of a traditional approach targeting single branches, with up to 62 percent smaller test suites.