An experimental evaluation of the assumption of independence in multiversion programming
IEEE Transactions on Software Engineering
PODS—A project on diverse software
IEEE Transactions on Software Engineering - Special issue on reliability and safety in real-time process control
The Consistent Comparison Problem in N-Version Software
IEEE Transactions on Software Engineering
Analysis of Faults in an N-Version Software Experiment
IEEE Transactions on Software Engineering
Program Transformation Systems
ACM Computing Surveys (CSUR)
Fail-stop processors: an approach to designing fault-tolerant computing systems
ACM Transactions on Computer Systems (TOCS)
The Science of Programming
Fault Tolerance: Principles and Practice
Fault Tolerance: Principles and Practice
A software quality assurance experiment
Proceedings of the software quality assurance workshop on Functional and performance issues
Testing software using multiple versions
Testing software using multiple versions
An Empirical Comparison of Software Fault Tolerance and Fault Elimination
IEEE Transactions on Software Engineering
Towards a method of programming with assertions
ICSE '92 Proceedings of the 14th international conference on Software engineering
Faults on its sleeve: amplifying software reliability testing
ISSTA '93 Proceedings of the 1993 ACM SIGSOFT international symposium on Software testing and analysis
A Practical Approach to Programming With Assertions
IEEE Transactions on Software Engineering
Assertion-oriented automated test data generation
Proceedings of the 18th international conference on Software engineering
A systematic approach for implementing fault tolerant software designs in Ada
TRI-Ada '90 Proceedings of the conference on TRI-ADA '90
Toward a resourceful method of software fault tolerance
ACM-SE 37 Proceedings of the 37th annual Southeast regional conference (CD-ROM)
Intent Specifications: An Approach to Building Human-Centered Specifications
IEEE Transactions on Software Engineering
Automatically Checking an Implementation against Its Formal Specification
IEEE Transactions on Software Engineering
Dynamically Discovering Likely Program Invariants to Support Program Evolution
IEEE Transactions on Software Engineering - Special issue on 1999 international conference on software engineering
On systematic design of globally consistent executable assertions in embedded software
Proceedings of the joint conference on Languages, compilers and tools for embedded systems: software and compilers for embedded systems
Software quality through domain-driven certification
Annals of Software Engineering
Component-Based Synthesis of Dependable Embedded Software
FTRTFT '02 Proceedings of the 7th International Symposium on Formal Techniques in Real-Time and Fault-Tolerant Systems: Co-sponsored by IFIP WG 2.2
Empirical Assessment of Software On-Line Diagnostics Using Fault Injection
SAFECOMP '00 Proceedings of the 19th International Conference on Computer Safety, Reliability and Security
Component Metadata for Software Engineering Tasks
EDO '00 Revised Papers from the Second International Workshop on Engineering Distributed Objects
Executable assertions and timed traces for on-line software error detection
FTCS '96 Proceedings of the The Twenty-Sixth Annual International Symposium on Fault-Tolerant Computing (FTCS '96)
Design of reliable software via general combination of N-version programming and acceptance testing
ISSRE '96 Proceedings of the The Seventh International Symposium on Software Reliability Engineering
Cost of Software Design Diversity: An Empirical Evaluation
ISSRE '99 Proceedings of the 10th International Symposium on Software Reliability Engineering
FTCS '95 Proceedings of the Twenty-Fifth International Symposium on Fault-Tolerant Computing
EPIC: Profiling the Propagation and Effect of Data Errors in Software
IEEE Transactions on Computers
Improving the adaptability of multi-mode systems via program steering
ISSTA '04 Proceedings of the 2004 ACM SIGSOFT international symposium on Software testing and analysis
Putting Detectors in Their Place
SEFM '05 Proceedings of the Third IEEE International Conference on Software Engineering and Formal Methods
A relational database model of program execution and software components
ACM-SE 38 Proceedings of the 38th annual on Southeast regional conference
A historical perspective on runtime assertion checking in software development
ACM SIGSOFT Software Engineering Notes
Proceedings of the 1st workshop on Architectural and system support for improving software dependability
The Daikon system for dynamic detection of likely invariants
Science of Computer Programming
An approach to synthesise safe systems
International Journal of Security and Networks
N-version programming with imperfect debugging
Computers and Electrical Engineering
Issues on the design of efficient fail-safe fault tolerance
ISSRE'09 Proceedings of the 20th IEEE international conference on software reliability engineering
An approach for designing and assessing detectors for dependable component-based systems
HASE'04 Proceedings of the Eighth IEEE international conference on High assurance systems engineering
Experiences with the design of a run-time check
SAFECOMP'06 Proceedings of the 25th international conference on Computer Safety, Reliability, and Security
On the effectiveness of run-time checks
SAFECOMP'05 Proceedings of the 24th international conference on Computer Safety, Reliability, and Security
Designing efficient fail-safe multitolerant systems
FORTE'05 Proceedings of the 25th IFIP WG 6.1 international conference on Formal Techniques for Networked and Distributed Systems
Dependable and Historic Computing
Checking formal specifications by testing
IW-FM'99 Proceedings of the 3rd Irish conference on Formal Methods
Verifying quantitative reliability for programs that execute on unreliable hardware
Proceedings of the 2013 ACM SIGPLAN international conference on Object oriented programming systems languages & applications
Hi-index | 0.01 |
The results of an empirical study of software error detection using self checks and N-version voting are presented. Working independently, each of 24 programmers first prepared a set of self checks using just the requirements specification of an aerospace application, and then each added self checks to an existing implementation of that specification. The modified programs were executed to measure the error-detection performance of the checks and to compare this with error detection using simple voting among multiple versions. The analysis of the checks revealed that there are great differences in the ability of individual programmers to design effective checks. It was found that some checks that might have been effective failed to detect an error because they were badly placed, and there were numerous instances of checks signaling nonexistent errors. In general, specification-based checks alone were not as effective as specification-based checks combined with code-based checks. Self checks made it possible to identify faults that had not been detected previously by voting 28 versions of the program over a million randomly generated inputs. This appeared to result from the fact that the self checks could examine the internal state of the executing program, whereas voting examines only final results of computations. If internal states had to be identical in N-version voting systems, then there would be no reason to write multiple versions.