Partition Testing Does Not Inspire Confidence (Program Testing)
IEEE Transactions on Software Engineering
Analyzing Partition Testing Strategies
IEEE Transactions on Software Engineering
Optimal Test Distributions for Software Failure Cost Estimation
IEEE Transactions on Software Engineering
Randomized algorithms
Partition Testing vs. Random Testing: The Influence of Uncertainty
IEEE Transactions on Software Engineering
Art of Software Testing
On some reliability estimation problems in random and partition testing
IEEE Transactions on Software Engineering
Comparing Partition and Random Testing via Majorization and Schur Functions
IEEE Transactions on Software Engineering
Search-based software test data generation: a survey: Research Articles
Software Testing, Verification & Reliability
On the statistical properties of testing effectiveness measures
Journal of Systems and Software - Special issue: Quality software
Some upper and lower bounds on the coupon collector problem
Journal of Computational and Applied Mathematics
Feedback-Directed Random Test Generation
ICSE '07 Proceedings of the 29th international conference on Software Engineering
Randomized Differential Testing as a Prelude to Formal Verification
ICSE '07 Proceedings of the 29th international conference on Software Engineering
A search-based framework for automatic testing of MATLAB/Simulink models
Journal of Systems and Software
Search based software testing of object-oriented containers
Information Sciences: an International Journal
ICSTW '10 Proceedings of the 2010 Third International Conference on Software Testing, Verification, and Validation Workshops
A Theoretical and Empirical Study of Search-Based Testing: Local, Global, and Hybrid Search
IEEE Transactions on Software Engineering
Theoretical analysis of local search in software testing
SAGA'09 Proceedings of the 5th international conference on Stochastic algorithms: foundations and applications
Longer is Better: On the Role of Test Sequence Length in Software Testing
ICST '10 Proceedings of the 2010 Third International Conference on Software Testing, Verification and Validation
On the number and nature of faults found by random testing
Software Testing, Verification & Reliability
A baseline method for search-based software engineering
Proceedings of the 6th International Conference on Predictive Models in Software Engineering
Black-box system testing of real-time embedded systems using random and search-based testing
ICTSS'10 Proceedings of the 22nd IFIP WG 6.1 international conference on Testing software and systems
Proceedings of the 33rd International Conference on Software Engineering
Testing container classes: random or systematic?
FASE'11/ETAPS'11 Proceedings of the 14th international conference on Fundamental approaches to software engineering: part of the joint European conferences on theory and practice of software
Adaptive random testing: an illusion of effectiveness?
Proceedings of the 2011 International Symposium on Software Testing and Analysis
Is branch coverage a good measure of testing effectiveness?
Empirical Software Engineering and Verification
Coverage rewarded: Test input generation via adaptation-based programming
ASE '11 Proceedings of the 2011 26th IEEE/ACM International Conference on Automated Software Engineering
Stateful testing: Finding more errors in code and contracts
ASE '11 Proceedings of the 2011 26th IEEE/ACM International Conference on Automated Software Engineering
On the danger of coverage directed test case generation
FASE'12 Proceedings of the 15th international conference on Fundamental Approaches to Software Engineering
Proceedings of the 2012 International Symposium on Software Testing and Analysis
CarFast: achieving higher statement coverage faster
Proceedings of the ACM SIGSOFT 20th International Symposium on the Foundations of Software Engineering
The search for the laws of automatic random testing
Proceedings of the 28th Annual ACM Symposium on Applied Computing
Scheduling black-box mutational fuzzing
Proceedings of the 2013 ACM SIGSAC conference on Computer & communications security
Hi-index | 0.00 |
There has been a lot of work to shed light on whether random testing is actually a useful testing technique. Despite its simplicity, several successful real-world applications appear in the literature. Although it is not going to solve all possible testing problems, random testing is an essential tool in the hands of software testers. In this paper, we address general questions about random testing, such as how long random testing needs on average to achieve testing targets (e.g., coverage), how does it scale and how likely is it to yield similar results if we re-run random testing on the same testing problem. Due to its simplicity that makes the mathematical analysis of random testing tractable, we provide precise and rigorous answers to these questions. Our formal results can be applied to most types of software and testing criteria. Simulations are carried out to provide further support to our formal results. The obtained results are then used to assess the validity of empirical analyses reported in the literature. Results show that random testing is more effective and predictable than previously thought.