Experiences with the design of a run-time check

Authors:
Meine J. P. van der Meulen;Miguel A. Revilla
Affiliations:
Centre for Software Reliability, City University, London, UK;University of Valladolid, Valladolid, Spain
Venue:
SAFECOMP'06 Proceedings of the 25th international conference on Computer Safety, Reliability, and Security
Year:
2006

Citing 8
Cited 0

The Use of Self Checks and Voting in Software Error Detection: An Empirical Study

IEEE Transactions on Software Engineering
Software reliability via run-time result-checking

Journal of the ACM (JACM)
Fault Tolerance: Principles and Practice

Fault Tolerance: Principles and Practice
Applying "Design by Contract"

Computer
Reducing Critical Failures for Control Algorithms Using Executable Assertions and Best Effort Recover

DSN '01 Proceedings of the 2001 International Conference on Dependable Systems and Networks (formerly: FTCS)
The Reliability of Diverse Systems: A Contribution Using Modelling of the Fault Creation Process

DSN '01 Proceedings of the 2001 International Conference on Dependable Systems and Networks (formerly: FTCS)
Experimental evaluation of the fail-silent behaviour in programs with consistency checks

FTCS '96 Proceedings of the The Twenty-Sixth Annual International Symposium on Fault-Tolerant Computing (FTCS '96)
On the effectiveness of run-time checks

SAFECOMP'05 Proceedings of the 24th international conference on Computer Safety, Reliability, and Security

Quantified Score

Hi-index	0.00

Visualization

Abstract

Run-time checks are often assumed to be a cost-effective way of improving the dependability of software components, by checking required properties of their outputs and flagging an output as incorrect if it fails the check. Run-time checks' main point of attractiveness is that they are supposed to be easy to implement. Also, they are implicitly assumed to be effective in detecting incorrect outputs. This paper reports the results of an experiment designed to challenge these assumptions about run-time checks. The experiment uses a subset of 196 of 867 programs (primaries) solving a problem called “Make Palindrome”. This is an existing problem on the “On-Line Judge” website of the university of Valladolid. We formulated eight run-time checks, and posted this problem on the same web-site. This resulted in 335 programs (checkers) implementing the run-time checks, 115 of which are used for the experiment. In this experiment: (1) the effectiveness of the population of possibly faulty checkers is very close to the effectiveness of a correct checker; (2) the reliability improvement provided by the run-time checks is relatively small, between a factor of one and three; (3) The reliability improvement gained by using multiple-version redundancy is much higher. Given the fact that this experiment only considers one primary/Run-Time Check combination, it is not yet possible to generalise the results.