An empirical study of the reliability of UNIX utilities
Communications of the ACM
Partition Testing Does Not Inspire Confidence (Program Testing)
IEEE Transactions on Software Engineering
The chaining approach for software test data generation
ACM Transactions on Software Engineering and Methodology (TOSEM)
On random and partition testing
Proceedings of the 1998 ACM SIGSOFT international symposium on Software testing and analysis
QuickCheck: a lightweight tool for random testing of Haskell programs
ICFP '00 Proceedings of the fifth ACM SIGPLAN international conference on Functional programming
Automated Software Engineering
Test input generation with java PathFinder
ISSTA '04 Proceedings of the 2004 ACM SIGSOFT international symposium on Software testing and analysis
Rostra: A Framework for Detecting Redundant Object-Oriented Unit Tests
Proceedings of the 19th IEEE international conference on Automated software engineering
JCrasher: an automatic robustness tester for Java
Software—Practice & Experience
Check 'n' crash: combining static checking and testing
Proceedings of the 27th international conference on Software engineering
DART: directed automated random testing
Proceedings of the 2005 ACM SIGPLAN conference on Programming language design and implementation
QSIC '05 Proceedings of the Fifth International Conference on Quality Software
Test input generation for java containers using state matching
Proceedings of the 2006 international symposium on Software testing and analysis
DSD-Crasher: a hybrid analysis tool for bug finding
Proceedings of the 2006 international symposium on Software testing and analysis
ASE '06 Proceedings of the 21st IEEE/ACM International Conference on Automated Software Engineering
An empirical study of the robustness of Windows NT applications using random testing
WSS'00 Proceedings of the 4th conference on USENIX Windows Systems Symposium - Volume 4
Jartege: a tool for random generation of unit tests for java classes
QoSA'05 Proceedings of the First international conference on Quality of Software Architectures and Software Quality, and Proceedings of the Second International conference on Software Quality
A theory of predicate-complete test coverage and generation
FMCO'04 Proceedings of the Third international conference on Formal Methods for Components and Objects
CUTE and jCUTE: concolic unit testing and explicit path model-checking tools
CAV'06 Proceedings of the 18th international conference on Computer Aided Verification
Symstra: a framework for generating object-oriented unit tests using symbolic execution
TACAS'05 Proceedings of the 11th international conference on Tools and Algorithms for the Construction and Analysis of Systems
Eclat: automatic generation and classification of test inputs
ECOOP'05 Proceedings of the 19th European conference on Object-Oriented Programming
Randomized Differential Testing as a Prelude to Formal Verification
ICSE '07 Proceedings of the 29th international conference on Software Engineering
State space exploration using feedback constraint generation and Monte-Carlo sampling
Proceedings of the the 6th joint meeting of the European software engineering conference and the ACM SIGSOFT symposium on The foundations of software engineering
Directed random reduction of combinatorial test suites
Proceedings of the 2nd international workshop on Random testing: co-located with the 22nd IEEE/ACM International Conference on Automated Software Engineering (ASE 2007)
Automated Software Engineering
Randoop: feedback-directed random testing for Java
Companion to the 22nd ACM SIGPLAN conference on Object-oriented programming systems and applications companion
Theory-infected: or how i learned to stop worrying and love universal quantification
Companion to the 22nd ACM SIGPLAN conference on Object-oriented programming systems and applications companion
Combined static and dynamic mutability analysis
Proceedings of the twenty-second IEEE/ACM international conference on Automated software engineering
Directed test generation using symbolic grammars
Proceedings of the twenty-second IEEE/ACM international conference on Automated software engineering
Nighthawk: a two-level genetic-random unit test data generator
Proceedings of the twenty-second IEEE/ACM international conference on Automated software engineering
Effective random testing of concurrent programs
Proceedings of the twenty-second IEEE/ACM international conference on Automated software engineering
A genetic approach for random testing of database systems
VLDB '07 Proceedings of the 33rd international conference on Very large data bases
Generating Java unit tests with AI planning
Proceedings of the 1st ACM international workshop on Empirical assessment of software engineering languages and technologies: held in conjunction with the 22nd IEEE/ACM International Conference on Automated Software Engineering (ASE) 2007
ARTOO: adaptive random testing for object-oriented software
Proceedings of the 30th international conference on Software engineering
Experience applying the SPIN model checker to an industrial telecommunications system
Proceedings of the 30th international conference on Software engineering
Grammar-based whitebox fuzzing
Proceedings of the 2008 ACM SIGPLAN conference on Programming language design and implementation
Finding errors in .net with feedback-directed random testing
ISSTA '08 Proceedings of the 2008 international symposium on Software testing and analysis
Random testing and model checking: building a common framework for nondeterministic exploration
WODA '08 Proceedings of the 2008 international workshop on dynamic analysis: held in conjunction with the ACM SIGSOFT International Symposium on Software Testing and Analysis (ISSTA 2008)
Automatically repairing event sequence-based GUI test suites for regression testing
ACM Transactions on Software Engineering and Methodology (TOSEM)
ReCrash: Making Software Failures Reproducible by Preserving Object States
ECOOP '08 Proceedings of the 22nd European conference on Object-Oriented Programming
Randomized directed testing (REDIRECT) for Simulink/Stateflow models
EMSOFT '08 Proceedings of the 8th ACM international conference on Embedded software
Automation of broad sanity test generation
Programming and Computing Software
Parameter reference immutability: formal definition, inference tool, and comparison
Automated Software Engineering
PROMISE '09 Proceedings of the 5th International Conference on Predictor Models in Software Engineering
Equality and hashing for (almost) free: Generating implementations from abstraction functions
ICSE '09 Proceedings of the 31st International Conference on Software Engineering
MSeqGen: object-oriented unit-test generation via mining source code
Proceedings of the the 7th joint meeting of the European software engineering conference and the ACM SIGSOFT symposium on The foundations of software engineering
Integration of verification methods for program systems
Programming and Computing Software
On the Use of Uniform Random Generation of Automata for Testing
Electronic Notes in Theoretical Computer Science (ENTCS)
Improving safety when refactoring aspect-oriented programs
Proceedings of the 24th ACM SIGPLAN conference companion on Object oriented programming systems languages and applications
Random Test Run Length and Effectiveness
ASE '08 Proceedings of the 2008 23rd IEEE/ACM International Conference on Automated Software Engineering
Predicting Effectiveness of Automatic Testing Tools
ASE '08 Proceedings of the 2008 23rd IEEE/ACM International Conference on Automated Software Engineering
Adaptive Random Testing: The ART of test case diversity
Journal of Systems and Software
State Based Robustness Testing for Components
Electronic Notes in Theoretical Computer Science (ENTCS)
Test case generation based on invariant extraction
WiCOM'09 Proceedings of the 5th International Conference on Wireless communications, networking and mobile computing
Iterative execution-feedback model-directed GUI testing
Information and Software Technology
Pex: white box test generation for .NET
TAP'08 Proceedings of the 2nd international conference on Tests and proofs
SEIM: static extraction of interaction models
Proceedings of the 2nd International Workshop on Principles of Engineering Service-Oriented Systems
Making program refactoring safer
Proceedings of the 32nd ACM/IEEE International Conference on Software Engineering - Volume 2
Automatically identifying critical input regions and code in applications
Proceedings of the 19th international symposium on Software testing and analysis
OCAT: object capture-based automated testing
Proceedings of the 19th international symposium on Software testing and analysis
Formal analysis of the effectiveness and predictability of random testing
Proceedings of the 19th international symposium on Software testing and analysis
Robustness testing for software components
Science of Computer Programming
Using virtual machines for collaborative research
Proceedings of the 2010 ACM-IEEE International Symposium on Empirical Software Engineering and Measurement
HotOS'09 Proceedings of the 12th conference on Hot topics in operating systems
Random unit-test generation with MUT-aware sequence recommendation
Proceedings of the IEEE/ACM international conference on Automated software engineering
Randomized constraint solvers: a comparative study
Innovations in Systems and Software Engineering
Random testing for higher-order, stateful programs
Proceedings of the ACM international conference on Object oriented programming systems languages and applications
A uniform random test data generator for path testing
Journal of Systems and Software
Rethinking the economics of software engineering
Proceedings of the FSE/SDP workshop on Future of software engineering research
Issues in testing collection class libraries
Proceedings of the 1st Workshop on Testing Object-Oriented Systems
DyGen: automatic generation of high-coverage tests via mining gigabytes of dynamic traces
TAP'10 Proceedings of the 4th international conference on Tests and proofs
Testing techniques in software engineering
Testing techniques in software engineering
Journal of Electronic Testing: Theory and Applications
Increasing functional coverage by inductive testing: a case study
ICTSS'10 Proceedings of the 22nd IFIP WG 6.1 international conference on Testing software and systems
FortressCheck: automatic testing for generic properties
Proceedings of the 2011 ACM Symposium on Applied Computing
Test data to reduce the complexity of unit test automation
Proceedings of the 6th International Workshop on Automation of Software Test
MeCC: memory comparison-based clone detector
Proceedings of the 33rd International Conference on Software Engineering
A framework for automated testing of javascript web applications
Proceedings of the 33rd International Conference on Software Engineering
GATE: game-based testing environment
Proceedings of the 33rd International Conference on Software Engineering
Proceedings of the 33rd International Conference on Software Engineering
Palus: a hybrid automated test generation tool for java
Proceedings of the 33rd International Conference on Software Engineering
NFM'11 Proceedings of the Third international conference on NASA Formal methods
Testing container classes: random or systematic?
FASE'11/ETAPS'11 Proceedings of the 14th international conference on Fundamental approaches to software engineering: part of the joint European conferences on theory and practice of software
Retrofitting unit tests for parameterized unit testing
FASE'11/ETAPS'11 Proceedings of the 14th international conference on Fundamental approaches to software engineering: part of the joint European conferences on theory and practice of software
Adaptive random testing: an illusion of effectiveness?
Proceedings of the 2011 International Symposium on Software Testing and Analysis
CoDeSe: fast deserialization via code generation
Proceedings of the 2011 International Symposium on Software Testing and Analysis
Combined static and dynamic automated test generation
Proceedings of the 2011 International Symposium on Software Testing and Analysis
Search-based software testing and test data generation for a dynamic programming language
Proceedings of the 13th annual conference on Genetic and evolutionary computation
Android permissions demystified
Proceedings of the 18th ACM conference on Computer and communications security
Synthesizing method sequences for high-coverage testing
Proceedings of the 2011 ACM international conference on Object oriented programming systems languages and applications
Safira: a tool for evaluating behavior preservation
Proceedings of the ACM international conference companion on Object oriented programming systems languages and applications companion
Automatic testing environment for multi-core embedded software-ATEMES
Journal of Systems and Software
(Quickly) testing the tester via path coverage
WODA '09 Proceedings of the Seventh International Workshop on Dynamic Analysis
Fast black-box testing of system recovery code
Proceedings of the 7th ACM european conference on Computer Systems
Coverage rewarded: Test input generation via adaptation-based programming
ASE '11 Proceedings of the 2011 26th IEEE/ACM International Conference on Automated Software Engineering
ASE '11 Proceedings of the 2011 26th IEEE/ACM International Conference on Automated Software Engineering
Automated documentation inference to explain failed tests
ASE '11 Proceedings of the 2011 26th IEEE/ACM International Conference on Automated Software Engineering
On generating mutants for AspectJ programs
Information and Software Technology
Fully automatic and precise detection of thread safety violations
Proceedings of the 33rd ACM SIGPLAN conference on Programming Language Design and Implementation
Proceedings of the 2012 International Symposium on Software Testing and Analysis
Isolating failure causes through test case generation
Proceedings of the 2012 International Symposium on Software Testing and Analysis
Finding common ground: choose, assert, and assume
Proceedings of the 2012 Workshop on Dynamic Analysis
Sound empirical evidence in software testing
Proceedings of the 34th International Conference on Software Engineering
Proceedings of the 34th International Conference on Software Engineering
BALLERINA: automatic generation and clustering of efficient random unit tests for multithreaded code
Proceedings of the 34th International Conference on Software Engineering
Augmenting test suites automatically
Proceedings of the 34th International Conference on Software Engineering
Puzzle-based automatic testing: bringing humans into the loop by solving puzzles
Proceedings of the 27th IEEE/ACM International Conference on Automated Software Engineering
PROFES'12 Proceedings of the 13th international conference on Product-Focused Software Process Improvement
Synthesizing iterators from abstraction functions
Proceedings of the 11th International Conference on Generative Programming and Component Engineering
Automated behavioral testing of refactoring engines
Proceedings of the 3rd annual conference on Systems, programming, and applications: software for humanity
Test input generation using dynamic programming
Proceedings of the ACM SIGSOFT 20th International Symposium on the Foundations of Software Engineering
Comparing approaches to analyze refactoring activity on software repositories
Journal of Systems and Software
Exposing behavioral differences in cross-language API mapping relations
FASE'13 Proceedings of the 16th international conference on Fundamental Approaches to Software Engineering
The search for the laws of automatic random testing
Proceedings of the 28th Annual ACM Symposium on Applied Computing
Comparing non-adequate test suites using coverage criteria
Proceedings of the 2013 International Symposium on Software Testing and Analysis
Feedback-directed unit test generation for C/C++ using concolic execution
Proceedings of the 2013 International Conference on Software Engineering
Comparing multi-point stride coverage and dataflow coverage
Proceedings of the 2013 International Conference on Software Engineering
Automatic testing of sequential and concurrent substitutability
Proceedings of the 2013 International Conference on Software Engineering
Automated diagnosis of software configuration errors
Proceedings of the 2013 International Conference on Software Engineering
Automatic patch generation learned from human-written patches
Proceedings of the 2013 International Conference on Software Engineering
Practical semantic test simplification
Proceedings of the 2013 International Conference on Software Engineering
Scheduling black-box mutational fuzzing
Proceedings of the 2013 ACM SIGSAC conference on Computer & communications security
PathART: path-sensitive adaptive random testing
Proceedings of the 5th Asia-Pacific Symposium on Internetware
Hi-index | 0.00 |
We present a technique that improves random test generation by incorporating feedback obtained from executing test inputs as they are created. Our technique builds inputs incrementally by randomly selecting a method call to apply and finding arguments from among previously-constructed inputs. As soon as an input is built, it is executed and checked against a set of contracts and filters. The result of the execution determines whether the input is redundant, illegal, contract-violating, or useful for generating more inputs. The technique outputs a test suite consisting of unit tests for the classes under test. Passing tests can be used to ensure that code contracts are preserved across program changes; failing tests (that violate one or more contract) point to potential errors that should be corrected. Our experimental results indicate that feedback-directed random test generation can outperform systematic and undirected random test generation, in terms of coverage and error detection. On four small but nontrivial data structures (used previously in the literature), our technique achieves higher or equal block and predicate coverage than model checking (with and without abstraction) and undirected random generation. On 14 large, widely-used libraries (comprising 780KLOC), feedback-directed random test generation finds many previously-unknown errors, not found by either model checking or undirected random generation.