Orthogonal Latin squares: an application of experiment design to compiler testing
Communications of the ACM
Applying design of experiments to software testing: experience report
ICSE '97 Proceedings of the 19th international conference on Software engineering
The AETG System: An Approach to Testing Based on Combinatorial Design
IEEE Transactions on Software Engineering
Model-based testing in practice
Proceedings of the 21st international conference on Software engineering
A practical guide to testing object-oriented software
A practical guide to testing object-oriented software
Software Engineering: A Practitioner's Approach
Software Engineering: A Practitioner's Approach
Software Fault Interactions and Implications for Software Testing
IEEE Transactions on Software Engineering
Covering arrays for efficient fault characterization in complex configuration spaces
ISSTA '04 Proceedings of the 2004 ACM SIGSOFT international symposium on Software testing and analysis
Experimental designs in software engineering: d-optimal designs and covering arrays
Proceedings of the 2004 ACM workshop on Interdisciplinary software engineering research
A framework of greedy methods for constructing interaction test suites
Proceedings of the 27th international conference on Software engineering
Proceedings of the 27th international conference on Software engineering
Test prioritization for pairwise interaction coverage
A-MOST '05 Proceedings of the 1st international workshop on Advances in model-based testing
Distributed performance testing using statistical modeling
A-MOST '05 Proceedings of the 1st international workshop on Advances in model-based testing
Covering Arrays for Efficient Fault Characterization in Complex Configuration Spaces
IEEE Transactions on Software Engineering
Empirical Software Engineering
Testing across configurations: implications for combinatorial testing
ACM SIGSOFT Software Engineering Notes
Combinatorial testing: learnings from our experience
ACM SIGSOFT Software Engineering Notes
IEEE Transactions on Software Engineering
One-test-at-a-time heuristic search for interaction test suites
Proceedings of the 9th annual conference on Genetic and evolutionary computation
Generating combinatorial test suite for interaction relationship
Fourth international workshop on Software quality assurance: in conjunction with the 6th ESEC/FSE joint meeting
Designing a combinatorial java unit testing tool
ACST'07 Proceedings of the third conference on IASTED International Conference: Advances in Computer Science and Technology
Configuration-aware regression testing: an empirical study of sampling and prioritization
ISSTA '08 Proceedings of the 2008 international symposium on Software testing and analysis
A backtracking search tool for constructing combinatorial test suites
Journal of Systems and Software
Covering Arrays Avoiding Forbidden Edges
COCOA 2008 Proceedings of the 2nd international conference on Combinatorial Optimization and Applications
The Effectiveness of T-Way Test Data Generation
SAFECOMP '08 Proceedings of the 27th international conference on Computer Safety, Reliability, and Security
Orthogonal array application for optimal combination of software defect detection techniques choices
WSEAS Transactions on Computers
Merging covering arrays and compressing multiple sequence alignments
Discrete Applied Mathematics
Combining Satisfiability Solving and Heuristics to Constrained Combinatorial Interaction Testing
TAP '09 Proceedings of the 3rd International Conference on Tests and Proofs
Covering arrays avoiding forbidden edges
Theoretical Computer Science
Algorithms to locate errors using covering arrays
LATIN'08 Proceedings of the 8th Latin American conference on Theoretical informatics
Comparing the effectiveness of testing techniques
Formal methods and testing
Using symbolic evaluation to understand behavior in configurable software systems
Proceedings of the 32nd ACM/IEEE International Conference on Software Engineering - Volume 1
A survey of combinatorial testing
ACM Computing Surveys (CSUR)
A Formal Logic Approach to Constrained Combinatorial Testing
Journal of Automated Reasoning
Advances in Software Engineering - Special issue on software test automation
Locating Errors Using ELAs, Covering Arrays, and Adaptive Testing Algorithms
SIAM Journal on Discrete Mathematics
Techniques to reduce a set of test cases
ICCOM'06 Proceedings of the 10th WSEAS international conference on Communications
The Minimal Failure-Causing Schema of Combinatorial Testing
ACM Transactions on Software Engineering and Methodology (TOSEM)
Characterizing failure-causing parameter interactions by adaptive testing
Proceedings of the 2011 International Symposium on Software Testing and Analysis
Hardness results for covering arrays avoiding forbidden edges and error-locating arrays
Theoretical Computer Science
A variable strength interaction test suites generation strategy using Particle Swarm Optimization
Journal of Systems and Software
New bounds for binary covering arrays using simulated annealing
Information Sciences: an International Journal
Model-based testing of industrial transformational systems
ICTSS'11 Proceedings of the 23rd IFIP WG 6.1 international conference on Testing software and systems
Industrial evaluation of pairwise SPL testing with MoSo-PoLiTe
Proceedings of the Sixth International Workshop on Variability Modeling of Software-Intensive Systems
Finding the best CAFE is NP-hard
LATIN'10 Proceedings of the 9th Latin American conference on Theoretical Informatics
An exact approach to maximize the number of wild cards in a covering array
MICAI'11 Proceedings of the 10th Mexican international conference on Advances in Artificial Intelligence - Volume Part I
SP 800-142. Practical Combinatorial Testing
SP 800-142. Practical Combinatorial Testing
iTree: efficiently discovering high-coverage configurations using interaction trees
Proceedings of the 34th International Conference on Software Engineering
T-wise combinatorial interaction test suites construction based on coverage inheritance
Software Testing, Verification & Reliability
An auto-adapted method to generate pairwise test data set
AICI'12 Proceedings of the 4th international conference on Artificial Intelligence and Computational Intelligence
MICAI'12 Proceedings of the 11th Mexican international conference on Advances in Computational Intelligence - Volume Part II
Evaluating applicability of combinatorial testing in an industrial environment: a case study
Proceedings of the 2013 International Workshop on Joining AcadeMiA and Industry Contributions to testing Automation
A general conformance testing framework for IEEE 11073 PHD's communication model
Proceedings of the 6th International Conference on PErvasive Technologies Related to Assistive Environments
Mixed optimization combinatorial method for constructing covering arrays
Programming and Computing Software
Hi-index | 0.00 |
Approaches to software testing based on methods from the field of design of experiments have been advocated as a means of providing high coverage at relatively low cost. Tools to generate all pairs, or higher n-degree combinations,of input values have been developed and demonstrated in a few applications, but little empirical evidence is available to aid developers in evaluating the effectiveness of these tools for particular problems. In this paper we investigate error reports from two large open-source software projects, a browser and web server, to provide preliminary answers to three questions: Is there a point of diminishing returns at which generating all n-degree combinations is nearly as effective as all n+1-degree combinations? What is the appropriate value of n for particular classes of software? Does this value differ for different types of software, and by how much? Our findings suggest that more than 95% of errors in the software studied would be detected by test cases that cover all 4-way combinations of values, and that the browser and server software were similar in the percentage of errors detectable by combinations of degree 2 through 6.