Efficiently computing static single assignment form and the control dependence graph
ACM Transactions on Programming Languages and Systems (TOPLAS)
Constraint-Based Automatic Test Data Generation
IEEE Transactions on Software Engineering
Automatic test data generation using constraint solving techniques
Proceedings of the 1998 ACM SIGSOFT international symposium on Software testing and analysis
A Discipline of Programming
Korat: automated testing based on Java predicates
ISSTA '02 Proceedings of the 2002 ACM SIGSOFT international symposium on Software testing and analysis
Generating Software Test Data by Evolution
IEEE Transactions on Software Engineering
Efficient management of XML contents over wireless environment by Xstream
Proceedings of the 2004 ACM symposium on Applied computing
Labeling images with a computer game
Proceedings of the SIGCHI Conference on Human Factors in Computing Systems
DART: directed automated random testing
Proceedings of the 2005 ACM SIGPLAN conference on Programming language design and implementation
CUTE: a concolic unit testing engine for C
Proceedings of the 10th European software engineering conference held jointly with 13th ACM SIGSOFT international symposium on Foundations of software engineering
Feedback-Directed Random Test Generation
ICSE '07 Proceedings of the 29th international conference on Software Engineering
A System to Generate Test Data and Symbolically Execute Programs
IEEE Transactions on Software Engineering
Calysto: scalable and precise extended static checking
Proceedings of the 30th international conference on Software engineering
Randomized active atomicity violation detection in concurrent programs
Proceedings of the 16th ACM SIGSOFT International Symposium on Foundations of software engineering
Snugglebug: a powerful approach to weakest preconditions
Proceedings of the 2009 ACM SIGPLAN conference on Programming language design and implementation
Accurate Interprocedural Null-Dereference Analysis for Java
ICSE '09 Proceedings of the 31st International Conference on Software Engineering
MSeqGen: object-oriented unit-test generation via mining source code
Proceedings of the the 7th joint meeting of the European software engineering conference and the ACM SIGSOFT symposium on The foundations of software engineering
Predicting Effectiveness of Automatic Testing Tools
ASE '08 Proceedings of the 2008 23rd IEEE/ACM International Conference on Automated Software Engineering
Pex: white box test generation for .NET
TAP'08 Proceedings of the 2nd international conference on Tests and proofs
The challenge of designing scientific discovery games
Proceedings of the Fifth International Conference on the Foundations of Digital Games
OCAT: object capture-based automated testing
Proceedings of the 19th international symposium on Software testing and analysis
Guided test generation for coverage criteria
ICSM '10 Proceedings of the 2010 IEEE International Conference on Software Maintenance
Test generation via Dynamic Symbolic Execution for mutation testing
ICSM '10 Proceedings of the 2010 IEEE International Conference on Software Maintenance
Precise identification of problems for structural test generation
Proceedings of the 33rd International Conference on Software Engineering
Symbolic execution with mixed concrete-symbolic solving
Proceedings of the 2011 International Symposium on Software Testing and Analysis
Optimizing Constraint Solving to Better Support Symbolic Execution
ICSTW '11 Proceedings of the 2011 IEEE Fourth International Conference on Software Testing, Verification and Validation Workshops
Pex4Fun: Teaching and learning computer science via social gaming
CSEET '11 Proceedings of the 2011 24th IEEE-CS Conference on Software Engineering Education and Training
Closing the gap between modelling and java
SLE'09 Proceedings of the Second international conference on Software Language Engineering
CUTE and jCUTE: concolic unit testing and explicit path model-checking tools
CAV'06 Proceedings of the 18th international conference on Computer Aided Verification
Verification games: making verification fun
Proceedings of the 14th Workshop on Formal Techniques for Java-like Programs
Hi-index | 0.00 |
Recently, many automatic test generation techniques have been proposed, such as Randoop, Pex and jCUTE. However, usually test coverage of these techniques has been around 50-60% only, due to several challenges, such as 1) the object mutation problem, where test generators cannot create and/or modify test inputs to desired object states; and 2) the constraint solving problem, where test generators fail to solve path conditions to cover certain branches. By analyzing branches not covered by state-of-the-art techniques, we noticed that these challenges might not be so difficult for humans. To verify this hypothesis, we propose a Puzzle-based Automatic Testing environment (PAT) which decomposes object mutation and complex constraint solving problems into small puzzles for humans to solve. We generated PAT puzzles for two open source projects and asked different groups of people to solve these puzzles. It was shown that they could be effectively solved by humans: 231 out of 400 puzzles were solved by humans at an average speed of one minute per puzzle. The 231 puzzle solutions helped cover 534 and 308 additional branches (7.0% and 5.8% coverage improvement) in the two open source projects, on top of the saturated branch coverages achieved by the two state-of-the-art test generation techniques.