An experimental evaluation of the assumption of independence in multiversion programming
IEEE Transactions on Software Engineering
An empirical study of the reliability of UNIX utilities
Communications of the ACM
C programming FAQs: frequently asked questions
C programming FAQs: frequently asked questions
Simplifying and Isolating Failure-Inducing Input
IEEE Transactions on Software Engineering
An integrated experimental environment for distributed systems and networks
OSDI '02 Proceedings of the 5th symposium on Operating systems design and implementationCopyright restrictions prevent ACM from being able to make the PDFs for this conference available for downloading
Random testing of C calling conventions
Proceedings of the sixth international symposium on Automated analysis-driven debugging
Cutpoints for formal equivalence verification of embedded software
Proceedings of the 5th ACM international conference on Embedded software
How compilers and tools differ for embedded systems
Proceedings of the 2005 international conference on Compilers, architectures and synthesis for embedded systems
HDD: hierarchical delta debugging
Proceedings of the 28th international conference on Software engineering
Practical testing of a C99 compiler using output comparison
Software—Practice & Experience
Grammar-based whitebox fuzzing
Proceedings of the 2008 ACM SIGPLAN conference on Programming language design and implementation
Volatiles are miscompiled, and what to do about it
EMSOFT '08 Proceedings of the 8th ACM international conference on Embedded software
A general test data generator for COBOL
AIEE-IRE '62 (Spring) Proceedings of the May 1-3, 1962, spring joint computer conference
Formal verification of a realistic compiler
Communications of the ACM - Barbara Liskov: ACM's A.M. Turing Award Winner
seL4: formal verification of an OS kernel
Proceedings of the ACM SIGOPS 22nd symposium on Operating systems principles
Automatic generation of test cases
IBM Systems Journal
Formal Verification of Avionics Software Products
FM '09 Proceedings of the 2nd World Congress on Formal Methods
Bringing extensibility to verified compilers
PLDI '10 Proceedings of the 2010 ACM SIGPLAN conference on Programming language design and implementation
ARMor: fully verified software fault isolation
EMSOFT '11 Proceedings of the ninth ACM international conference on Embedded software
Integrated language definition testing: enabling test-driven language development
Proceedings of the 2011 ACM international conference on Object oriented programming systems languages and applications
An executable formal semantics of C with applications
POPL '12 Proceedings of the 39th annual ACM SIGPLAN-SIGACT symposium on Principles of programming languages
Test-case reduction for C compiler bugs
Proceedings of the 33rd ACM SIGPLAN conference on Programming Language Design and Implementation
RockSalt: better, faster, stronger SFI for the x86
Proceedings of the 33rd ACM SIGPLAN conference on Programming Language Design and Implementation
Testing static analyzers with randomly generated programs
NFM'12 Proceedings of the 4th international conference on NASA Formal Methods
Proceedings of the 2012 International Symposium on Software Testing and Analysis
Evaluating program analysis and testing tools with the RUGRAT random benchmark application generator
Proceedings of the 2012 Workshop on Dynamic Analysis
Finding common ground: choose, assert, and assume
Proceedings of the 2012 Workshop on Dynamic Analysis
An OpenMP 3.1 validation testsuite
IWOMP'12 Proceedings of the 8th international conference on OpenMP in a Heterogeneous World
Establishing browser security guarantees through formal shim verification
Security'12 Proceedings of the 21st USENIX conference on Security symposium
Security'12 Proceedings of the 21st USENIX conference on Security symposium
Experience report: a do-it-yourself high-assurance compiler
Proceedings of the 17th ACM SIGPLAN international conference on Functional programming
Test input generation using dynamic programming
Proceedings of the ACM SIGSOFT 20th International Symposium on the Foundations of Software Engineering
Learning-Based test programming for programmers
ISoLA'12 Proceedings of the 5th international conference on Leveraging Applications of Formal Methods, Verification and Validation: technologies for mastering change - Volume Part I
An executable semantics for compcert c
CPP'12 Proceedings of the Second international conference on Certified Programs and Proofs
Formal verification of SSA-based optimizations for LLVM
Proceedings of the 34th ACM SIGPLAN conference on Programming language design and implementation
Proceedings of the 34th ACM SIGPLAN conference on Programming language design and implementation
Translation validation for a verified OS kernel
Proceedings of the 34th ACM SIGPLAN conference on Programming language design and implementation
Abstract interpretation of microcontroller code: Intervals meet congruences
Science of Computer Programming
Fuzzing the ActionScript virtual machine
Proceedings of the 8th ACM SIGSAC symposium on Information, computer and communications security
Will you still compile me tomorrow? static cross-version compiler validation
Proceedings of the 2013 9th Joint Meeting on Foundations of Software Engineering
Compiler testing via a theory of sound optimisations in the C11/C++11 memory model
Proceedings of the 34th ACM SIGPLAN conference on Programming language design and implementation
Testing noninterference, quickly
Proceedings of the 18th ACM SIGPLAN international conference on Functional programming
Steering symbolic execution to less traveled paths
Proceedings of the 2013 ACM SIGPLAN international conference on Object oriented programming systems languages & applications
Strato: a retargetable framework for low-level inlined-reference monitors
SEC'13 Proceedings of the 22nd USENIX conference on Security
Runtime verification of microcontroller binary code
Science of Computer Programming
Hi-index | 0.00 |
Compilers should be correct. To improve the quality of C compilers, we created Csmith, a randomized test-case generation tool, and spent three years using it to find compiler bugs. During this period we reported more than 325 previously unknown bugs to compiler developers. Every compiler we tested was found to crash and also to silently generate wrong code when presented with valid input. In this paper we present our compiler-testing tool and the results of our bug-hunting study. Our first contribution is to advance the state of the art in compiler testing. Unlike previous tools, Csmith generates programs that cover a large subset of C while avoiding the undefined and unspecified behaviors that would destroy its ability to automatically find wrong-code bugs. Our second contribution is a collection of qualitative and quantitative results about the bugs we have found in open-source C compilers.