Dependability of COTS Microkernel-Based Systems
IEEE Transactions on Computers - Special issue on fault-tolerant embedded systems
Fault Injection Techniques and Tools
Computer
Software Dependability in the Tandem GUARDIAN System
IEEE Transactions on Software Engineering
Xception: A Technique for the Experimental Evaluation of Dependability in Modern Computers
IEEE Transactions on Software Engineering
Comparing the Robustness of POSIX Operating Systems
FTCS '99 Proceedings of the Twenty-Ninth Annual International Symposium on Fault-Tolerant Computing
Failure Data Analysis of a LAN of Windows NT Based Computers
SRDS '99 Proceedings of the 18th IEEE Symposium on Reliable Distributed Systems
A dependability benchmark for OLTP application environments
VLDB '03 Proceedings of the 29th international conference on Very large data bases - Volume 29
From Performance to Dependability Benchmarking: A Mandatory Path
Performance Evaluation and Benchmarking
Issues in Benchmark Metric Selection
Performance Evaluation and Benchmarking
Energy benchmarks: a detailed analysis
Proceedings of the 1st International Conference on Energy-Efficient Computing and Networking
Dependable computing: concepts, limits, challenges
FTCS'95 Proceedings of the Twenty-Fifth international conference on Fault-tolerant computing
Extending TPC-E to measure availability in database systems
TPCTC'11 Proceedings of the Third TPC Technology conference on Topics in Performance Evaluation, Measurement and Characterization
| Hi-index | 0.00 |
Transactional systems are the core of the information systems of most organizations. Although there is general acknowledgement that failures in these systems often entail significant impact both on the proceeds and reputation of companies, the benchmarks developed and managed by the Transaction Processing Performance Council (TPC) still maintain their focus on reporting bare performance. Each TPC benchmark has to pass a list of dependability-related tests (to verify ACID properties), but not all benchmarks require measuring their performances. While TPC-E measures the recovery time of some system failures, TPC-H and TPC-C only require functional correctness of such recovery. Consequently, systems used in TPC benchmarks are tuned mostly for performance. In this paper we argue that nowadays systems should be tuned for a more comprehensive suite of dependability tests, and that a dependability metric should be part of TPC benchmark publications. The paper discusses WHY and HOW this can be achieved. Two approaches are introduced and discussed: augmenting each TPC benchmark in a customized way, by extending each specification individually; and pursuing a more unified approach, defining a generic specification that could be adjoined to any TPC benchmark.