An empirical study of the reliability of UNIX utilities
Communications of the ACM
The synchronization of periodic routing messages
IEEE/ACM Transactions on Networking (TON)
DART: directed automated random testing
Proceedings of the 2005 ACM SIGPLAN conference on Programming language design and implementation
EXE: automatically generating inputs of death
Proceedings of the 13th ACM conference on Computer and communications security
Compositional dynamic test generation
Proceedings of the 34th annual ACM SIGPLAN-SIGACT symposium on Principles of programming languages
The Security Development Lifecycle
The Security Development Lifecycle
Framework for instruction-level tracing and analysis of program executions
Proceedings of the 2nd international conference on Virtual execution environments
Automatically classifying benign and harmful data races using replay analysis
Proceedings of the 2007 ACM SIGPLAN conference on Programming language design and implementation
An empirical study of the robustness of Windows NT applications using random testing
WSS'00 Proceedings of the 4th conference on USENIX Windows Systems Symposium - Volume 4
TACAS'08/ETAPS'08 Proceedings of the Theory and practice of software, 14th international conference on Tools and algorithms for the construction and analysis of systems
Pex: white box test generation for .NET
TAP'08 Proceedings of the 2nd international conference on Tests and proofs
KLEE: unassisted and automatic generation of high-coverage tests for complex systems programs
OSDI'08 Proceedings of the 8th USENIX conference on Operating systems design and implementation
Dynamic test generation to find integer bugs in x86 binary linux programs
SSYM'09 Proceedings of the 18th conference on USENIX security symposium
S2E: a platform for in-vivo multi-path analysis of software systems
Proceedings of the sixteenth international conference on Architectural support for programming languages and operating systems
Parallel symbolic execution for automated real-world software testing
Proceedings of the sixth conference on Computer systems
Symbolic execution for software testing in practice: preliminary assessment
Proceedings of the 33rd International Conference on Software Engineering
Proceedings of the 32nd ACM SIGPLAN conference on Programming language design and implementation
Statically validating must summaries for incremental compositional dynamic test generation
SAS'11 Proceedings of the 18th international conference on Static analysis
Scheduling black-box mutational fuzzing
Proceedings of the 2013 ACM SIGSAC conference on Computer & communications security
Hi-index | 0.00 |
We report experiences with constraint-based whitebox fuzz testing in production across hundreds of large Windows applications and over 500 machine years of computation from 2007 to 2013. Whitebox fuzzing leverages symbolic execution on binary traces and constraint solving to construct new inputs to a program. These inputs execute previously uncovered paths or trigger security vulnerabilities. Whitebox fuzzing has found one-third of all file fuzzing bugs during the development of Windows 7, saving millions of dollars in potential security vulnerabilities. The technique is in use today across multiple products at Microsoft. We describe key challenges with running whitebox fuzzing in production. We give principles for addressing these challenges and describe two new systems built from these principles: SAGAN, which collects data from every fuzzing run for further analysis, and JobCenter, which controls deployment of our whitebox fuzzing infrastructure across commodity virtual machines. Since June 2010, SAGAN has logged over 3.4 billion constraints solved, millions of symbolic executions, and tens of millions of test cases generated. Our work represents the largest scale deployment of whitebox fuzzing to date, including the largest usage ever for a Satisfiability Modulo Theories (SMT) solver. We present specific data analyses that improved our production use of whitebox fuzzing. Finally we report data on the performance of constraint solving and dynamic test generation that points toward future research problems.