Taming compiler fuzzers

Authors:
Yang Chen;Alex Groce;Chaoqiang Zhang;Weng-Keen Wong;Xiaoli Fern;Eric Eide;John Regehr
Affiliations:
University of Utah, Salt Lake City, UT, USA;Oregon State University, Corvallis, OR, USA;Oregon State University, Corvallis, OR, USA;Oregon State University, Corvallis, OR, USA;Oregon State University, Corvallis, OR, USA;University of Utah, Salt Lake City, UT, USA;University of Utah, Salt Lake City, UT, USA
Venue:
Proceedings of the 34th ACM SIGPLAN conference on Programming language design and implementation
Year:
2013

Citing 34
Cited 0

Automatic isolation of compiler errors

ACM Transactions on Programming Languages and Systems (TOPLAS)
QuickCheck: a lightweight tool for random testing of Haskell programs

ICFP '00 Proceedings of the fifth ACM SIGPLAN international conference on Functional programming
A vector space model for automatic indexing

Communications of the ACM
Simplifying and Isolating Failure-Inducing Input

IEEE Transactions on Software Engineering
Visualization of test information to assist fault localization

Proceedings of the 24th International Conference on Software Engineering
X-means: Extending K-means with Efficient Estimation of the Number of Clusters

ICML '00 Proceedings of the Seventeenth International Conference on Machine Learning
Automated support for classifying software failure reports

Proceedings of the 25th International Conference on Software Engineering
Bug isolation via remote program sampling

PLDI '03 Proceedings of the ACM SIGPLAN 2003 conference on Programming language design and implementation
Winnowing: local algorithms for document fingerprinting

Proceedings of the 2003 ACM SIGMOD international conference on Management of data
Cluster ensembles --- a knowledge reuse framework for combining multiple partitions

The Journal of Machine Learning Research
Explaining abstract counterexamples

Proceedings of the 12th ACM SIGSOFT twelfth international symposium on Foundations of software engineering
Tree-Based Methods for Classifying Software Failures

ISSRE '04 Proceedings of the 15th International Symposium on Software Reliability Engineering
Locating causes of program failures

Proceedings of the 27th international conference on Software engineering
Scalable statistical bug isolation

Proceedings of the 2005 ACM SIGPLAN conference on Programming language design and implementation
Empirical evaluation of the tarantula automatic fault-localization technique

Proceedings of the 20th IEEE/ACM international Conference on Automated software engineering
Failure proximity: a fault localization-based approach

Proceedings of the 14th ACM SIGSOFT international symposium on Foundations of software engineering
Randomized Differential Testing as a Prelude to Formal Verification

ICSE '07 Proceedings of the 29th international conference on Software Engineering
Valgrind: a framework for heavyweight dynamic binary instrumentation

Proceedings of the 2007 ACM SIGPLAN conference on Programming language design and implementation
Debugging in Parallel

Proceedings of the 2007 international symposium on Software testing and analysis
Active sampling for multiple output identification

Machine Learning
An approach to detecting duplicate bug reports using natural language and execution information

Proceedings of the 30th international conference on Software engineering
Category detection using hierarchical mean shift

Proceedings of the 15th ACM SIGKDD international conference on Knowledge discovery and data mining
Test case comparison and clustering using program profiles and static execution

Proceedings of the the 7th joint meeting of the European software engineering conference and the ACM SIGSOFT symposium on The foundations of software engineering
Random Test Run Length and Effectiveness

ASE '08 Proceedings of the 2008 23rd IEEE/ACM International Conference on Automated Software Engineering
Z-ranking: using statistical analysis to counter the impact of static analysis approximations

SAS'03 Proceedings of the 10th international conference on Static analysis
A discriminative model approach for accurate duplicate bug report retrieval

Proceedings of the 32nd ACM/IEEE International Conference on Software Engineering - Volume 1
Finding and understanding bugs in C compilers

Proceedings of the 32nd ACM SIGPLAN conference on Programming language design and implementation
Malware Variant Detection Using Similarity Search over Sets of Control Flow Graphs

TRUSTCOM '11 Proceedings of the 2011IEEE 10th International Conference on Trust, Security and Privacy in Computing and Communications
Taming false alarms from a domain-unaware c analyzer by a bayesian statistical post analysis

SAS'05 Proceedings of the 12th international conference on Static Analysis
Towards more accurate retrieval of duplicate bug reports

ASE '11 Proceedings of the 2011 26th IEEE/ACM International Conference on Automated Software Engineering
Test-case reduction for C compiler bugs

Proceedings of the 33rd ACM SIGPLAN conference on Programming Language Design and Implementation
Swarm testing

Proceedings of the 2012 International Symposium on Software Testing and Analysis
An empirical study about the effectiveness of debugging when random test cases are used

Proceedings of the 34th International Conference on Software Engineering
Fuzzing with code fragments

Security'12 Proceedings of the 21st USENIX conference on Security symposium

Quantified Score

Hi-index	0.00

Visualization

Abstract

Aggressive random testing tools ("fuzzers") are impressively effective at finding compiler bugs. For example, a single test-case generator has resulted in more than 1,700 bugs reported for a single JavaScript engine. However, fuzzers can be frustrating to use: they indiscriminately and repeatedly find bugs that may not be severe enough to fix right away. Currently, users filter out undesirable test cases using ad hoc methods such as disallowing problematic features in tests and grepping test results. This paper formulates and addresses the fuzzer taming problem: given a potentially large number of random test cases that trigger failures, order them such that diverse, interesting test cases are highly ranked. Our evaluation shows our ability to solve the fuzzer taming problem for 3,799 test cases triggering 46 bugs in a C compiler and 2,603 test cases triggering 28 bugs in a JavaScript engine.