Comparing the Effectiveness of Software Testing Strategies
IEEE Transactions on Software Engineering
An experimental comparison of the effectiveness of the all-uses and all-edges adequacy criteria
TAV4 Proceedings of the symposium on Testing, analysis, and verification
Assessing the fault-detecting ability of testing methods
SIGSOFT '91 Proceedings of the conference on Software for citical systems
Black-box testing: techniques for functional testing of software and systems
Black-box testing: techniques for functional testing of software and systems
A semantic model of program faults
ISSTA '96 Proceedings of the 1996 ACM SIGSOFT international symposium on Software testing and analysis
Further empirical studies of test effectiveness
SIGSOFT '98/FSE-6 Proceedings of the 6th ACM SIGSOFT international symposium on Foundations of software engineering
Analyzing exception flow in Java programs
ESEC/FSE-7 Proceedings of the 7th European software engineering conference held jointly with the 7th ACM SIGSOFT international symposium on Foundations of software engineering
Hierarchical GUI Test Case Generation Using Automated Planning
IEEE Transactions on Software Engineering - Special issue on 1999 international conference on software engineering
An Empirical Analysis of Equivalence Partitioning, Boundary Value Analysis and Random Testing
METRICS '97 Proceedings of the 4th International Symposium on Software Metrics
Static analysis to support the evolution of exception structure in object-oriented systems
ACM Transactions on Software Engineering and Methodology (TOSEM)
JCrasher: an automatic robustness tester for Java
Software—Practice & Experience
Is mutation an appropriate tool for testing experiments?
Proceedings of the 27th international conference on Software engineering
Discovering faults in idiom-based exception handling
Proceedings of the 28th international conference on Software engineering
An empirical analysis and comparison of random testing techniques
Proceedings of the 2006 ACM/IEEE international symposium on Empirical software engineering
Software Testing Research: Achievements, Challenges, Dreams
FOSE '07 2007 Future of Software Engineering
An event-flow model of GUI-based applications for testing: Research Articles
Software Testing, Verification & Reliability
International Journal on Software Tools for Technology Transfer (STTT)
Automatically repairing event sequence-based GUI test suites for regression testing
ACM Transactions on Software Engineering and Methodology (TOSEM)
Using Probabilistic Model Checking to Evaluate GUI Testing Techniques
SEFM '09 Proceedings of the 2009 Seventh IEEE International Conference on Software Engineering and Formal Methods
Automated Behavioral Regression Testing
ICST '10 Proceedings of the 2010 Third International Conference on Software Testing, Verification and Validation
Fault Detection Likelihood of Test Sequence Length
ICST '10 Proceedings of the 2010 Third International Conference on Software Testing, Verification and Validation
Proceedings of the 33rd International Conference on Software Engineering
Accounting for defect characteristics in evaluations of testing techniques
ACM Transactions on Software Engineering and Methodology (TOSEM)
An Evaluation of Random Testing
IEEE Transactions on Software Engineering
Amplifying tests to validate exception handling code
Proceedings of the 34th International Conference on Software Engineering
| Hi-index | 0.00 |
Software testing is a labor-intensive task in software development life-cycle. Human knowledge is useful in the practices of software testing, especially GUI testing. There are many strategies for GUI testing assisted by human knowledge, in which manual random testing and manual functional testing are two of widely used ones. In this paper, an empirical study is conducted to compare random testing and functional testing in order to provide guidelines for GUI testing. 234 participants were recruited to create thousands of random and functional test cases for open source GUI applications. Some of these test cases were selected with certain coverage criteria and then run on GUI applications to evaluate random testing and functional testing. We study three aspects on the two testing strategies: effectiveness, complementarity and impact of test case length. Some useful observations in the empirical study are: (1) Random testing is more effective in the early stage of testing on small applications and functional testing has more extensive applicability for testing large sized applications. (2) Random testing and functional testing exhibit some complementarity in our experiment. (3) Short test cases can reveal some faults more quickly and long test cases can reveal more faults lastingly.