Testing static analyzers with randomly generated programs

Authors:
Pascal Cuoq;Benjamin Monate;Anne Pacalet;Virgile Prevosto;John Regehr;Boris Yakobowski;Xuejun Yang
Affiliations:
CEA, LIST, France;CEA, LIST, France;INRIA Sophia-Antipolis, France;CEA, LIST, France;University of Utah, United States;CEA, LIST, France;University of Utah, United States
Venue:
NFM'12 Proceedings of the 4th international conference on NASA Formal Methods
Year:
2012

Citing 3
Cited 3

Formal methods: Practice and experience

ACM Computing Surveys (CSUR)
Fuzzing and delta-debugging SMT solvers

Proceedings of the 7th International Workshop on Satisfiability Modulo Theories
Finding and understanding bugs in C compilers

Proceedings of the 32nd ACM SIGPLAN conference on Programming language design and implementation

Test-case reduction for C compiler bugs

Proceedings of the 33rd ACM SIGPLAN conference on Programming Language Design and Implementation
Evaluating program analysis and testing tools with the RUGRAT random benchmark application generator

Proceedings of the 2012 Workshop on Dynamic Analysis
Behind the scenes in SANTE: a combination of static and dynamic analyses

Automated Software Engineering

Quantified Score

Hi-index	0.00

Visualization

Abstract

Static analyzers should be correct. We used the random C-program generator Csmith, initially intended to test C compilers, to test parts of the Frama-C static analysis platform. Although Frama-C was already relatively mature at that point, fifty bugs were found and fixed during the process, in the front-end (AST elaboration and type-checking) and in the value analysis, constant propagation and slicing plug-ins. Several bugs were also found in Csmith, even though it had been extensively tested and had been used to find numerous bugs in compilers.