An empirical study of the reliability of UNIX utilities
Communications of the ACM
Practical Unix and Internet security (2nd ed.)
Practical Unix and Internet security (2nd ed.)
Using production grammars in software testing
Proceedings of the 2nd conference on Domain-specific languages
Art of Software Testing
Testing the Robustness of Windows NT Software
ISSRE '98 Proceedings of the The Ninth International Symposium on Software Reliability Engineering
Verifying a Multiprocessor Cache Controller Using Random Case
Verifying a Multiprocessor Cache Controller Using Random Case
Integrated TCP/IP Protocol Software Testing for Vulnerability Detection
ICCNMC '03 Proceedings of the 2003 International Conference on Computer Networks and Mobile Computing
Revolution in The Valley (hardcover)
Revolution in The Valley (hardcover)
SCL: a language for security testing of network applications
CASCON '05 Proceedings of the 2005 conference of the Centre for Advanced Studies on Collaborative research
An empirical study of the robustness of Windows NT applications using random testing
WSS'00 Proceedings of the 4th conference on USENIX Windows Systems Symposium - Volume 4
Accelerating software development through collaboration
Proceedings of the 24th International Conference on Software Engineering
Adaptive random testing through iterative partitioning revisited
Proceedings of the 3rd international workshop on Software quality assurance
Testing across configurations: implications for combinatorial testing
ACM SIGSOFT Software Engineering Notes
High coverage detection of input-related security facults
SSYM'03 Proceedings of the 12th conference on USENIX Security Symposium - Volume 12
Directed test generation using symbolic grammars
Proceedings of the twenty-second IEEE/ACM international conference on Automated software engineering
Information Assurance: Dependability and Security in Networked Systems
Information Assurance: Dependability and Security in Networked Systems
Callgraph properties of executables
AI Communications - Network Analysis in Natural Sciences and Engineering
TFTP vulnerability finding technique based on fuzzing
Computer Communications
Efficient file fuzz testing using automated analysis of binary file format
Journal of Systems Architecture: the EUROMICRO Journal
Extended program invariants: applications in testing and fault localization
Proceedings of the 2012 Workshop on Dynamic Analysis
Hi-index | 0.00 |
We report on the fourth in a series of studies on the reliability of application programs in the face of random input. Over the previous 15 years, we have studied the reliability of UNIX command line and X-Window based (GUI) applications and Windows applications. In this study, we apply our fuzz testing techniques to applications running on the Mac OS X operating system. We continue to use a simple, or even simplistic technique: unstructured black-box random testing, considering a failure to be a crash or hang. As in the previous three studies, the technique is crude but seems to be effective in locating bugs in real programs.We tested the reliability of 135 command-line UNIX utilities and thirty graphical applications on Mac OS X by feeding random input to each. We report on application failures -- crashes (dumps core) or hangs (loops indefinitely) -- and, where source code is available, we identify the causes of these failures and categorize them.Our testing crashed only 7% of the command-line utilities, a considerably lower rate of failure than observed in almost all cases of previous studies. We found the GUI-based applications to be less reliable: of the thirty that we tested, only eight did not crash or hang. Twenty others crashed, and two hung. These GUI results were noticeably worse than either of the previous Windows (Win32) or UNIX (X-Windows) studies.