Combining Satisfiability Solving and Heuristics to Constrained Combinatorial Interaction Testing
TAP '09 Proceedings of the 3rd International Conference on Tests and Proofs
Interaction Coverage Meets Path Coverage by SMT Constraint Solving
TESTCOM '09/FATES '09 Proceedings of the 21st IFIP WG 6.1 International Conference on Testing of Software and Communication Systems and 9th International FATES Workshop
Combinatorial test design in practice
Proceedings of the 32nd ACM/IEEE International Conference on Software Engineering - Volume 2
A platform for search-based testing of concurrent software
Proceedings of the 8th Workshop on Parallel and Distributed Systems: Testing, Analysis, and Debugging
A survey of combinatorial testing
ACM Computing Surveys (CSUR)
A Formal Logic Approach to Constrained Combinatorial Testing
Journal of Automated Reasoning
Generating combinatorial test cases by efficient SAT encodings suitable for CDCL SAT solvers
LPAR'10 Proceedings of the 17th international conference on Logic for programming, artificial intelligence, and reasoning
Evaluating improvements to a meta-heuristic search for constrained interaction testing
Empirical Software Engineering
Tuple density: a new metric for combinatorial test suites (NIER track)
Proceedings of the 33rd International Conference on Software Engineering
Pairwise testing of dynamic composite services
Proceedings of the 6th International Symposium on Software Engineering for Adaptive and Self-Managing Systems
A survey of methods for constructing covering arrays
Programming and Computing Software
The Minimal Failure-Causing Schema of Combinatorial Testing
ACM Transactions on Software Engineering and Methodology (TOSEM)
Feedback driven adaptive combinatorial testing
Proceedings of the 2011 International Symposium on Software Testing and Analysis
A variable strength interaction test suites generation strategy using Particle Swarm Optimization
Journal of Systems and Software
Properties of realistic feature models make combinatorial testing of product lines feasible
Proceedings of the 14th international conference on Model driven engineering languages and systems
Regression testing minimization, selection and prioritization: a survey
Software Testing, Verification & Reliability
A survey on software product line testing
Proceedings of the 16th International Software Product Line Conference - Volume 1
An algorithm for generating t-wise covering arrays from large feature models
Proceedings of the 16th International Software Product Line Conference - Volume 1
T-wise combinatorial interaction test suites construction based on coverage inheritance
Software Testing, Verification & Reliability
Bow tie testing: a testing pattern for product lines
Proceedings of the 16th European Conference on Pattern Languages of Programs
Generating better partial covering arrays by modeling weights on sub-product lines
MODELS'12 Proceedings of the 15th international conference on Model Driven Engineering Languages and Systems
Automatic selection of test execution plans from a video conferencing system product line
Proceedings of the VARiability for You Workshop: Variability Modeling Made Useful for Everyone
Using feature model knowledge to speed up the generation of covering arrays
Proceedings of the Seventh International Workshop on Variability Modelling of Software-intensive Systems
Towards efficient SPL testing by variant reduction
Proceedings of the 4th international workshop on Variability & composition
Proceedings of the 2013 9th Joint Meeting on Foundations of Software Engineering
Practical pairwise testing for software product lines
Proceedings of the 17th International Software Product Line Conference
An orchestrated survey of methodologies for automated software test case generation
Journal of Systems and Software
CAiSE'13 Proceedings of the 25th international conference on Advanced Information Systems Engineering
Constructing test cases for n-wise testing from tree-based test models
Proceedings of the Fourth Symposium on Information and Communication Technology
| Hi-index | 0.00 |
Researchers have explored the application of combinatorial interaction testing (CIT) methods to construct samples to drive systematic testing of software system configurations. Applying CIT to highly-configurable software systems is complicated by the fact that, in many such systems, there are constraints between specific configuration parameters that render certain combinations invalid. Many CIT algorithms lack a mechanism to avoid these. In recent work, automated constraint solving methods have been combined with search-based CIT construction methods to address the constraint problem with promising results. However, these techniques can incur a non-trivial overhead. In this paper, we build upon our previous work to develop a family of greedy CIT sample generation algorithms that exploit calculations made by modern boolean satisfiability (SAT) solvers to prune the search space of the CIT problem. We perform a comparative evaluation of the cost-effectiveness of these algorithms on four real-world highly-configurable software systems and on a population of synthetic examples that share the characteristics of those systems. In combination our techniques reduce the cost of CIT in the presence of constraints to 30\% of the cost of widely-used unconstrained CIT methods without sacrificing the quality of the solutions.