An empirical study of the reliability of UNIX utilities
Communications of the ACM
Simplifying and Isolating Failure-Inducing Input
IEEE Transactions on Software Engineering
CIL: Intermediate Language and Tools for Analysis and Transformation of C Programs
CC '02 Proceedings of the 11th International Conference on Compiler Construction
CMC: a pragmatic approach to model checking real code
OSDI '02 Proceedings of the 5th symposium on Operating systems design and implementationCopyright restrictions prevent ACM from being able to make the PDFs for this conference available for downloading
Minimization of Randomized Unit Test Cases
ISSRE '05 Proceedings of the 16th IEEE International Symposium on Software Reliability Engineering
A historical perspective on runtime assertion checking in software development
ACM SIGSOFT Software Engineering Notes
Statistical debugging: simultaneous identification of multiple bugs
ICML '06 Proceedings of the 23rd international conference on Machine learning
Feedback-Directed Random Test Generation
ICSE '07 Proceedings of the 29th international conference on Software Engineering
Using model checking to find serious file system errors
OSDI'04 Proceedings of the 6th conference on Symposium on Opearting Systems Design & Implementation - Volume 6
Spin model checker, the: primer and reference manual
Spin model checker, the: primer and reference manual
Towards a Framework for Differential Unit Testing of Object-Oriented Programs
AST '07 Proceedings of the Second International Workshop on Automation of Software Test
Nighthawk: a two-level genetic-random unit test data generator
Proceedings of the twenty-second IEEE/ACM international conference on Automated software engineering
Testing for buffer overflows with length abstraction
ISSTA '08 Proceedings of the 2008 international symposium on Software testing and analysis
Finding errors in .net with feedback-directed random testing
ISSTA '08 Proceedings of the 2008 international symposium on Software testing and analysis
Random testing and model checking: building a common framework for nondeterministic exploration
WODA '08 Proceedings of the 2008 international workshop on dynamic analysis: held in conjunction with the ACM SIGSOFT International Symposium on Software Testing and Analysis (ISSTA 2008)
BERT: BEhavioral Regression Testing
WODA '08 Proceedings of the 2008 international workshop on dynamic analysis: held in conjunction with the ACM SIGSOFT International Symposium on Software Testing and Analysis (ISSTA 2008)
Formal Modeling and Analysis of a Flash Filesystem in Alloy
ABZ '08 Proceedings of the 1st international conference on Abstract State Machines, B and Z
ASE '08 Proceedings of the 2008 23rd IEEE/ACM International Conference on Automated Software Engineering
Random Test Run Length and Effectiveness
ASE '08 Proceedings of the 2008 23rd IEEE/ACM International Conference on Automated Software Engineering
Unit Testing of Flash Memory Device Driver through a SAT-Based Model Checker
ASE '08 Proceedings of the 2008 23rd IEEE/ACM International Conference on Automated Software Engineering
RAFFS: Model Checking a Robust Abstract Flash File Store
ICFEM '09 Proceedings of the 11th International Conference on Formal Engineering Methods: Formal Methods and Software Engineering
Extending model checking with dynamic analysis
VMCAI'08 Proceedings of the 9th international conference on Verification, model checking, and abstract interpretation
Testing system virtual machines
Proceedings of the 19th international symposium on Software testing and analysis
Formal analysis of the effectiveness and predictability of random testing
Proceedings of the 19th international symposium on Software testing and analysis
HotOS'09 Proceedings of the 12th conference on Hot topics in operating systems
Property-based testing: the ProTest project
FMCO'09 Proceedings of the 8th international conference on Formal methods for components and objects
(Quickly) testing the tester via path coverage
WODA '09 Proceedings of the Seventh International Workshop on Dynamic Analysis
Proceedings of the 2012 International Symposium on Software Testing and Analysis
Finding common ground: choose, assert, and assume
Proceedings of the 2012 Workshop on Dynamic Analysis
Learning-Based test programming for programmers
ISoLA'12 Proceedings of the 5th international conference on Leveraging Applications of Formal Methods, Verification and Validation: technologies for mastering change - Volume Part I
Proceedings of the 34th ACM SIGPLAN conference on Programming language design and implementation
Comparing non-adequate test suites using coverage criteria
Proceedings of the 2013 International Symposium on Software Testing and Analysis
| Hi-index | 0.00 |
Most flight software testing at the Jet Propulsion Laboratory relies on the use of hand-produced test scenarios and is executed on systems as similar as possible to actual mission hardware. We report on a flight software development effort incorporating large-scale (biased) randomized testing on commodity desktop hardware. The results show that use of a reference implementation, hardware simulation with fault injection, a testable design, and test minimization enabled a high degree of automation in fault detection and correction. Our experience will be of particular interest to developers working in domains where on-time delivery of software is critical (a strong argument for randomized automated testing) but not at the expense of correctness and reliability (a strong argument for model checking, theorem proving, and other heavyweight techniques). The effort spent in randomized testing can prepare the way for generating more complete confidence using heavyweight techniques.