Orthogonal Latin squares: an application of experiment design to compiler testing
Communications of the ACM
Applying design of experiments to software testing: experience report
ICSE '97 Proceedings of the 19th international conference on Software engineering
The AETG System: An Approach to Testing Based on Combinatorial Design
IEEE Transactions on Software Engineering
Model-based testing in practice
Proceedings of the 21st international conference on Software engineering
Data mining: practical machine learning tools and techniques with Java implementations
Data mining: practical machine learning tools and techniques with Java implementations
Constructing test suites for interaction testing
Proceedings of the 25th International Conference on Software Engineering
An Investigation of the Applicability of Design of Experiments to Software Testing
SEW '02 Proceedings of the 27th Annual NASA Goddard Software Engineering Workshop (SEW-27'02)
Skoll: Distributed Continuous Quality Assurance
Proceedings of the 26th International Conference on Software Engineering
Experimental designs in software engineering: d-optimal designs and covering arrays
Proceedings of the 2004 ACM workshop on Interdisciplinary software engineering research
A framework of greedy methods for constructing interaction test suites
Proceedings of the 27th international conference on Software engineering
Proceedings of the 27th international conference on Software engineering
Profiling Deployed Software: Assessing Strategies and Testing Opportunities
IEEE Transactions on Software Engineering
Applying classification techniques to remotely-collected program execution data
Proceedings of the 10th European software engineering conference held jointly with 13th ACM SIGSOFT international symposium on Foundations of software engineering
Test prioritization for pairwise interaction coverage
A-MOST '05 Proceedings of the 1st international workshop on Advances in model-based testing
Distributed performance testing using statistical modeling
A-MOST '05 Proceedings of the 1st international workshop on Advances in model-based testing
Covering Arrays for Efficient Fault Characterization in Complex Configuration Spaces
IEEE Transactions on Software Engineering
Testing across configurations: implications for combinatorial testing
ACM SIGSOFT Software Engineering Notes
Using GUI Run-Time State as Feedback to Generate Test Cases
ICSE '07 Proceedings of the 29th international conference on Software Engineering
IEEE Transactions on Software Engineering
Techniques for Classifying Executions of Deployed Software to Support Software Engineering Tasks
IEEE Transactions on Software Engineering
Time will tell: fault localization using time spectra
Proceedings of the 30th international conference on Software engineering
A variable strength interaction test suites generation strategy using Particle Swarm Optimization
Journal of Systems and Software
Automated cookie collection testing
ACM Transactions on Software Engineering and Methodology (TOSEM)
Hi-index | 0.00 |
Testing systems with large configurations spaces that change often is a challenging problem. The cost and complexity of QA explodes because often there isn't just one system, but a multitude of related systems. Bugs may appear in certain configurations, but not in others.The Skoll system and process has been developed to test these types of systems through distributed, continuous quality assurance, leveraging user resources around-the-world, around-the-clock. It has been shown to be effective in automatically characterizing configurations in which failures manifest. The derived information helps developers quickly narrow down the cause of failures which then improves turn around time for fixes. However, this method does not scale well. It requires one to exhaustively test each configuration in the configuration space.In this paper we examine an alternative approach. The idea is to systematically sample the configuration space, test only the selected configurations, and conduct fault characterization on the resulting data. The sampling approach we use is based on calculating a mathematical object called a covering array. We empirically assess the effect of using covering array derived test schedules on the resulting fault characterizations and provide guidelines to practitioners for their use.