Comparative language fuzz testing: programming languages vs. fat fingers

Authors:
Diomidis Spinellis;Vassilios Karakoidas;Panos Louridas
Affiliations:
Athens University of Economics and Business, Athens, Greece;Athens University of Economics and Business, Athens, Greece;Athens University of Economics and Business, Athens, Greece
Venue:
Proceedings of the ACM 4th annual workshop on Evaluation and usability of programming languages and tools
Year:
2012

Citing 26
Cited 0

Collecting and categorizing software error data in an industrial environment

Journal of Systems and Software - Special issue on the fifth Minnowbrook workshop on software performance evaluation
The errors of TEX

Software—Practice & Experience
An empirical study of the reliability of UNIX utilities

Communications of the ACM
Computer related risks

Computer related risks
Types and programming languages

Types and programming languages
Software Fault Tolerance

Software Fault Tolerance
Godel, Escher, Bach: An Eternal Golden Braid

Godel, Escher, Bach: An Eternal Golden Braid
An Empirical Comparison of Seven Programming Languages

Computer
An analysis of errors and their causes in system programs

Proceedings of the international conference on Reliable software
Economic perspectives in test automation: balancing automated and manual testing with opportunity cost

Proceedings of the 2006 international workshop on Automation of software test
When only random testing will do

Proceedings of the 1st international workshop on Random testing
Fault-Tolerant Systems

Fault-Tolerant Systems
Searching for type-error messages

Proceedings of the 2007 ACM SIGPLAN conference on Programming language design and implementation
Automated testing of refactoring engines

Proceedings of the the 6th joint meeting of the European software engineering conference and the ACM SIGSOFT symposium on The foundations of software engineering
Random testing for security: blackbox vs. whitebox fuzzing

Proceedings of the 2nd international workshop on Random testing: co-located with the 22nd IEEE/ACM International Conference on Automated Software Engineering (ASE 2007)
Random testing: from the classical approach to a global view and full test automation

Proceedings of the 2nd international workshop on Random testing: co-located with the 22nd IEEE/ACM International Conference on Automated Software Engineering (ASE 2007)
Grammar-based whitebox fuzzing

Proceedings of the 2008 ACM SIGPLAN conference on Programming language design and implementation
Fuzzing for Software Security Testing and Quality Assurance

Fuzzing for Software Security Testing and Quality Assurance
Taint-based directed whitebox fuzzing

ICSE '09 Proceedings of the 31st International Conference on Software Engineering
Faith, hope, and love: an essay on software science's neglect of human factors

Proceedings of the ACM international conference on Object oriented programming systems languages and applications
Checksum-Aware Fuzzing Combined with Dynamic Taint Analysis and Symbolic Execution

ACM Transactions on Information and System Security (TISSEC)
Static vs. dynamic type systems: an empirical study about the relationship between type casts and development time

Proceedings of the 7th symposium on Dynamic languages
Tool-supported refactoring for JavaScript

Proceedings of the 2011 ACM international conference on Object oriented programming systems languages and applications
An Analysis and Survey of the Development of Mutation Testing

IEEE Transactions on Software Engineering
H-fuzzing: a new heuristic method for fuzzing data generation

NPC'11 Proceedings of the 8th IFIP international conference on Network and parallel computing
Refactoring tools for dynamic languages

Proceedings of the Fifth Workshop on Refactoring Tools

Quantified Score

Hi-index	0.00

Visualization

Abstract

We explore how programs written in ten popular programming languages are affected by small changes of their source code. This allows us to analyze the extend to which these languages allow the detection of simple errors at compile or at run time. Our study is based on a diverse corpus of programs written in several programming languages systematically perturbed using a mutation-based fuzz generator. The results we obtained prove that languages with weak type systems are significantly likelier than languages that enforce strong typing to let fuzzed programs compile and run, and, in the end, produce erroneous results. More importantly, our study also demonstrates the potential of comparative language fuzz testing for evaluating programming language designs.