Orthogonal Latin squares: an application of experiment design to compiler testing
Communications of the ACM
Applying design of experiments to software testing: experience report
ICSE '97 Proceedings of the 19th international conference on Software engineering
The AETG System: An Approach to Testing Based on Combinatorial Design
IEEE Transactions on Software Engineering
Factor-covering designs for testing software
Technometrics
Model-based testing in practice
Proceedings of the 21st international conference on Software engineering
Black-box test reduction using input-output analysis
Proceedings of the 2000 ACM SIGSOFT international symposium on Software testing and analysis
A new method for constructing pair-wise covering designs for software testing
Information Processing Letters
A Test Generation Strategy for Pairwise Testing
IEEE Transactions on Software Engineering
A Practical System for Mutation Testing: Help for the Common Programmer
Proceedings of the IEEE International Test Conference on TEST: The Next 25 Years
Determination of Test Configurations for Pair-Wise Interaction Coverage
TestCom '00 Proceedings of the IFIP TC6/WG6.1 13th International Conference on Testing Communicating Systems: Tools and Techniques
Formulation of the Interaction Test Coverage Problem as an Integer Program
TestCom '02 Proceedings of the IFIP 14th International Conference on Testing Communicating Systems XIV
Constructing test suites for interaction testing
Proceedings of the 25th International Conference on Software Engineering
Generating Expected Results for Automated Black-Box Testing
Proceedings of the 17th IEEE international conference on Automated software engineering
An Investigation of the Applicability of Design of Experiments to Software Testing
SEW '02 Proceedings of the 27th Annual NASA Goddard Software Engineering Workshop (SEW-27'02)
AETGSM Web: A Web Based Service for Automatic Efficient Test Generation from Functional Requirements
WIFT '98 Proceedings of the Second IEEE Workshop on Industrial Strength Formal Specification Techniques
A practical strategy for testing pair-wise coverage of network interfaces
ISSRE '96 Proceedings of the The Seventh International Symposium on Software Reliability Engineering
Model-based Testing of a Highly Programmable System
ISSRE '98 Proceedings of the The Ninth International Symposium on Software Reliability Engineering
Generating Test Cases for GUI Responsibilities Using Complete Interaction Sequences
ISSRE '00 Proceedings of the 11th International Symposium on Software Reliability Engineering
A Measure for Component Interaction Test Coverage
AICCSA '01 Proceedings of the ACS/IEEE International Conference on Computer Systems and Applications
A software testing model: using design of experiments (doe) and logistic regression
A software testing model: using design of experiments (doe) and logistic regression
Black-box test reduction using input-output analysis
Black-box test reduction using input-output analysis
Software component interaction testing: coverage measurement and generation of configurations
Software component interaction testing: coverage measurement and generation of configurations
Efficient Evaluation of Multifactor Dependent System Performance Using Fractional Factorial Design
IEEE Transactions on Software Engineering
Variable Strength Interaction Testing of Components
COMPSAC '03 Proceedings of the 27th Annual International Conference on Computer Software and Applications
Generating Small Combinatorial Test Suites to Cover Input-Output Relationships
QSIC '03 Proceedings of the Third International Conference on Quality Software
Using Spanning Sets for Coverage Testing
IEEE Transactions on Software Engineering
Augmenting Simulated Annealing to Build Interaction Test Suites
ISSRE '03 Proceedings of the 14th International Symposium on Software Reliability Engineering
Software Fault Interactions and Implications for Software Testing
IEEE Transactions on Software Engineering
Combinatorial group testing methods for the BIST diagnosis problem
Proceedings of the 2004 Asia and South Pacific Design Automation Conference
Comparing the Fault Detection Effectiveness of N-way and Random Test Suites
ISESE '04 Proceedings of the 2004 International Symposium on Empirical Software Engineering
Using Artificial Life Techniques to Generate Test Cases for Combinatorial Testing
COMPSAC '04 Proceedings of the 28th Annual International Computer Software and Applications Conference - Volume 01
A framework of greedy methods for constructing interaction test suites
Proceedings of the 27th international conference on Software engineering
Test prioritization for pairwise interaction coverage
A-MOST '05 Proceedings of the 1st international workshop on Advances in model-based testing
Modeling requirements for combinatorial software testing
A-MOST '05 Proceedings of the 1st international workshop on Advances in model-based testing
Constructing interaction test suites with greedy algorithms
Proceedings of the 20th IEEE/ACM international Conference on Automated software engineering
Improved Test Generation Algorithms for Pair-Wise Testing
ISSRE '05 Proceedings of the 16th IEEE International Symposium on Software Reliability Engineering
Covering Arrays for Efficient Fault Characterization in Complex Configuration Spaces
IEEE Transactions on Software Engineering
Constraint Models for the Covering Test Problem
Constraints
Algorithms for covering arrays
Algorithms for covering arrays
Backtracking Algorithms and Search Heuristics to Generate Test Suites for Combinatorial Testing
COMPSAC '06 Proceedings of the 30th Annual International Computer Software and Applications Conference - Volume 01
An evaluation of combination strategies for test case selection
Empirical Software Engineering
Generating Optimal Test Set for Neighbor Factors Combinatorial Testing
QSIC '06 Proceedings of the Sixth International Conference on Quality Software
Testing across configurations: implications for combinatorial testing
ACM SIGSOFT Software Engineering Notes
Combinatorial testing: learnings from our experience
ACM SIGSOFT Software Engineering Notes
IPOG: A General Strategy for T-Way Software Testing
ECBS '07 Proceedings of the 14th Annual IEEE International Conference and Workshops on the Engineering of Computer-Based Systems
The Current State and Future of Search Based Software Engineering
FOSE '07 2007 Future of Software Engineering
Search Algorithms for Regression Test Case Prioritization
IEEE Transactions on Software Engineering
Interaction testing of highly-configurable systems in the presence of constraints
Proceedings of the 2007 international symposium on Software testing and analysis
One-test-at-a-time heuristic search for interaction test suites
Proceedings of the 9th annual conference on Genetic and evolutionary computation
The density algorithm for pairwise interaction testing: Research Articles
Software Testing, Verification & Reliability
Towards incremental adaptive covering arrays
Proceedings of the the 6th joint meeting of the European software engineering conference and the ACM SIGSOFT symposium on The foundations of software engineering
Test suite prioritization by interaction coverage
Workshop on Domain specific approaches to software test automation: in conjunction with the 6th ESEC/FSE joint meeting
Generating combinatorial test suite for interaction relationship
Fourth international workshop on Software quality assurance: in conjunction with the 6th ESEC/FSE joint meeting
Exploiting Constraint Solving History to Construct Interaction Test Suites
TAICPART-MUTATION '07 Proceedings of the Testing: Academic and Industrial Conference Practice and Research Techniques - MUTATION
An Effective Iterative Metamorphic Testing Algorithm Based on Program Path Analysis
QSIC '07 Proceedings of the Seventh International Conference on Quality Software
A combinatorial testing strategy for concurrent programs
Software Testing, Verification & Reliability
Input parameter modeling for combination strategies
SE'07 Proceedings of the 25th conference on IASTED International Multi-Conference: Software Engineering
Prioritizing User-Session-Based Test Cases for Web Applications Testing
ICST '08 Proceedings of the 2008 International Conference on Software Testing, Verification, and Validation
IPOG-IPOG-D: efficient test generation for multi-way combinatorial testing
Software Testing, Verification & Reliability
IEEE Transactions on Software Engineering
Greedy Heuristic Algorithms to Generate Variable Strength Combinatorial Test Suite
QSIC '08 Proceedings of the 2008 The Eighth International Conference on Quality Software
A density-based greedy algorithm for higher strength covering arrays
Software Testing, Verification & Reliability
Upper bounds for covering arrays by tabu search
Discrete Applied Mathematics
A browser compatibility testing method based on combinatorial testing
ICWE'03 Proceedings of the 2003 international conference on Web engineering
A software debugging method based on pairwise testing
ICCS'05 Proceedings of the 5th international conference on Computational Science - Volume Part III
Automatic test generation for n-way combinatorial testing
QoSA'05 Proceedings of the First international conference on Quality of Software Architectures and Software Quality, and Proceedings of the Second International conference on Software Quality
Constraint-Based approaches to the covering test problem
CSCLP'04 Proceedings of the 2004 joint ERCIM/CoLOGNET international conference on Recent Advances in Constraints
Characterizing failure-causing parameter interactions by adaptive testing
Proceedings of the 2011 International Symposium on Software Testing and Analysis
Hardness results for covering arrays avoiding forbidden edges and error-locating arrays
Theoretical Computer Science
A variable strength interaction test suites generation strategy using Particle Swarm Optimization
Journal of Systems and Software
Properties of realistic feature models make combinatorial testing of product lines feasible
Proceedings of the 14th international conference on Model driven engineering languages and systems
Evolutionary algorithm for prioritized pairwise test data generation
Proceedings of the 14th annual conference on Genetic and evolutionary computation
Proceedings of the 2012 International Symposium on Software Testing and Analysis
Combining model-based and combinatorial testing for effective test case generation
Proceedings of the 2012 International Symposium on Software Testing and Analysis
Faulty interaction identification via constraint solving and optimization
SAT'12 Proceedings of the 15th international conference on Theory and Applications of Satisfiability Testing
Strengthening hash families and compressive sensing
Journal of Discrete Algorithms
Using feature model knowledge to speed up the generation of covering arrays
Proceedings of the Seventh International Workshop on Variability Modelling of Software-intensive Systems
MICAI'12 Proceedings of the 11th Mexican international conference on Advances in Computational Intelligence - Volume Part II
Proceedings of the 2013 9th Joint Meeting on Foundations of Software Engineering
Scalable analysis of variable software
Proceedings of the 2013 9th Joint Meeting on Foundations of Software Engineering
An orchestrated survey of methodologies for automated software test case generation
Journal of Systems and Software
Automated cookie collection testing
ACM Transactions on Software Engineering and Methodology (TOSEM)
Hi-index | 0.00 |
Combinatorial Testing (CT) can detect failures triggered by interactions of parameters in the Software Under Test (SUT) with a covering array test suite generated by some sampling mechanisms. It has been an active field of research in the last twenty years. This article aims to review previous work on CT, highlights the evolution of CT, and identifies important issues, methods, and applications of CT, with the goal of supporting and directing future practice and research in this area. First, we present the basic concepts and notations of CT. Second, we classify the research on CT into the following categories: modeling for CT, test suite generation, constraints, failure diagnosis, prioritization, metric, evaluation, testing procedure and the application of CT. For each of the categories, we survey the motivation, key issues, solutions, and the current state of research. Then, we review the contribution from different research groups, and present the growing trend of CT research. Finally, we recommend directions for future CT research, including: (1) modeling for CT, (2) improving the existing test suite generation algorithm, (3) improving analysis of testing result, (4) exploring the application of CT to different levels of testing and additional types of systems, (5) conducting more empirical studies to fully understand limitations and strengths of CT, and (6) combining CT with other testing techniques.