Robustness Testing of the Microsoft Win32 API

  • Authors:
  • Charles P. Shelton;Philip Koopman;Kobey Devale

  • Affiliations:
  • -;-;-

  • Venue:
  • DSN '00 Proceedings of the 2000 International Conference on Dependable Systems and Networks (formerly FTCS-30 and DCCA-8)
  • Year:
  • 2000

Quantified Score

Hi-index 0.00

Visualization

Abstract

Although Microsoft Windows is being deployed in mission-critical applications, little quantitative data has been published about its robustness. We present the results of executing over two million Ballista-generated exception handling tests across 237 functions and system calls involving six Windows variants, as well as similar tests conducted on the Linux operating system. Windows 95, Windows 98, and Windows CE were found to be vulnerable to complete system crashes caused by very simple C programs for several different functions. No system crashes were observed on Windows NT, Windows 2000, and Linux. Linux was significantly more graceful at handling exceptions from system calls in a program-recoverable manner than Windows NT and Windows 2000, but those Windows variants were more robust than Linux (with glibc) at handling C library exceptions. While the choice of operating systems cannot be made solely based on one set of tests, it is hoped that such results will form a starting point for comparing dependability across heterogeneous platforms.