Input parameter modeling for combination strategies
SE'07 Proceedings of the 25th conference on IASTED International Multi-Conference: Software Engineering
Comparing the effectiveness of testing techniques
Formal methods and testing
Clustering test cases to achieve effective test selection
Proceedings of the 1st Amrita ACM-W Celebration on Women in Computing in India
A survey of combinatorial testing
ACM Computing Surveys (CSUR)
The Minimal Failure-Causing Schema of Combinatorial Testing
ACM Transactions on Software Engineering and Methodology (TOSEM)
Characterizing failure-causing parameter interactions by adaptive testing
Proceedings of the 2011 International Symposium on Software Testing and Analysis
Testing techniques selection based on ODC fault types and software metrics
Journal of Systems and Software
Interaction-based test-suite minimization
Proceedings of the 2013 International Conference on Software Engineering
| Hi-index | 0.00 |
This paper presents results from a comparative evaluation of five combination strategies. Combination strategies are test case selection methods that combine "interesting" values of the input parameters of a test subject to form test cases. This research comparatively evaluated five combination strategies; the All Combination strategy (AC), the Each Choice strategy (EC), the Base Choice strategy (BC), Orthogonal Arrays (OA) and the algorithm from the Automatic Efficient Test Generator (AETG). AC satisfies n-wise coverage, EC and BC satisfy 1-wise coverage, and OA and AETG satisfy pair-wise coverage. The All Combinations strategy was used as a "gold standard" strategy; it subsumes the others but is usually too expensive for practical use. The others were used in an experiment that used five programs seeded with 128 faults. The combination strategies were evaluated with respect to the number of test cases, the number of faults found, failure size, and number of decisions covered. The strategy that requires the least number of tests, Each Choice, found the smallest number of faults. Although the Base Choice strategy requires fewer test cases than Orthogonal Arrays and AETG, it found as many faults. Analysis also shows some properties of the combination strategies that appear significant. The two most important results are that the Each Choice strategy is unpredictable in terms of which faults will be revealed, possibly indicating that faults are found by chance, and that the Base Choice and the pair-wise combination strategies to some extent target different types of faults.