Meaningful metrics for evaluating eventual consistency

  • Authors:
  • João Barreto;Paulo Ferreira

  • Affiliations:
  • INESC-ID, Technical University Lisbon, Lisboa, Portugal;INESC-ID, Technical University Lisbon, Lisboa, Portugal

  • Venue:
  • Euro-Par'10 Proceedings of the 16th international Euro-Par conference on Parallel processing: Part II
  • Year:
  • 2010

Quantified Score

Hi-index 0.00

Visualization

Abstract

Optimistic replication is a fundamental technique for supporting collaborative work practices in mobile environments. However, eventual consistency, in contrast to immediate strong consistency in pessimistic replication, is much harder to evaluate. This paper analyzes different metrics for measuring the effectiveness of eventually consistent systems. Using results from a simulated environment of relevant optimistic replication protocols, we show that each metric hides previously undocumented side effects. These add considerable imprecision to any evaluation that exclusively relies on a single metric. Hence, we advocate a combined methodology comprising three complementary metrics: commit ratio, average agreement delay and average commit delay.