Graph-Based Algorithms for Boolean Function Manipulation
IEEE Transactions on Computers
Improving the Variable Ordering of OBDDs Is NP-Complete
IEEE Transactions on Computers
The AETG System: An Approach to Testing Based on Combinatorial Design
IEEE Transactions on Software Engineering
Model-based testing in practice
Proceedings of the 21st international conference on Software engineering
Model checking
A Test Generation Strategy for Pairwise Testing
IEEE Transactions on Software Engineering
Determination of Test Configurations for Pair-Wise Interaction Coverage
TestCom '00 Proceedings of the IFIP TC6/WG6.1 13th International Conference on Testing Communicating Systems: Tools and Techniques
Constructing test suites for interaction testing
Proceedings of the 25th International Conference on Software Engineering
A Measure for Component Interaction Test Coverage
AICCSA '01 Proceedings of the ACS/IEEE International Conference on Computer Systems and Applications
Software Fault Interactions and Implications for Software Testing
IEEE Transactions on Software Engineering
Interaction testing of highly-configurable systems in the presence of constraints
Proceedings of the 2007 international symposium on Software testing and analysis
Practical Combinatorial Testing: Beyond Pairwise
IT Professional
Soa: principles of service design
Soa: principles of service design
Upper bounds for covering arrays by tabu search
Discrete Applied Mathematics
A logic-based approach to combinatorial testing with constraints
TAP'08 Proceedings of the 2nd international conference on Tests and proofs
Proceedings of the 2012 Workshop on Parallel and Distributed Systems: Testing, Analysis, and Debugging
Interactive refinement of combinatorial test plans
Proceedings of the 34th International Conference on Software Engineering
Interaction-based test-suite minimization
Proceedings of the 2013 International Conference on Software Engineering
An orchestrated survey of methodologies for automated software test case generation
Journal of Systems and Software
Hi-index | 0.04 |
Combinatorial test design (CTD) is an effective test planning technique that reveals faulty feature interaction in a given system. The test space is modeled by a set of parameters, their respective values, and restrictions on the value combinations. A subset of the test space is then automatically constructed so that it covers all valid value combinations of every t parameters, where t is a user input. Various combinatorial testing tools exist, implementing different approaches to finding a set of tests that satisfies t-wise coverage. However, little consideration has been given to the process of defining the test space for CTD, which is usually a manual, labor-intensive, and error-prone effort. Potential errors include missing parameters and their values, wrong identification of parameters and of valid value combinations, and errors in the definition of restrictions that cause them not to capture the intended combinations. From our experience, lack of support for the test space definition process is one of the main obstacles in applying CTD to a wide range of testing domains. In this work, we present a Cartesian product based methodology and technology that assist in defining a complete and consistent test space for CTD. We then show how using binary decision diagrams (BDDs) to represent and build the test space dramatically increases the scalability of our approach, making it applicable to large and complex real-life test design tasks, for which explicit representation of the test space is infeasible. Finally, we show how BDDs can be used also to solve the CTD problem itself. We present a new and highly effective BDD-based approach for solving CTD, which finds a set of tests that satisfies t-wise coverage by subset selection. Our approach supports also advanced requirements such as requirements on the distribution of values in the selected tests. We apply our algorithm to real-life testing problems of varying complexity, and show its superior performance.