Fast and Reliable Stream Processing over Wide Area Networks

Authors:
Jeong-Hyon Hwang;Ugur Cetintemel;Stan Zdonik
Affiliations:
Department of Computer Science, Box 1910, Brown University, Providence, RI 02912, USA. Tel: +1-401-863-6192, jhhwang@cs.brown.edu;Department of Computer Science, Box 1910, Brown University, Providence, RI 02912, USA. Tel: +1-401-863-6192, ugur@cs.brown.edu;Department of Computer Science, Box 1910, Brown University, Providence, RI 02912, USA. Tel: +1-401-863-6192, sbz@cs.brown.edu
Venue:
ICDEW '07 Proceedings of the 2007 IEEE 23rd International Conference on Data Engineering Workshop
Year:
2007

Citing 0
Cited 4

Load Management and High Availability in the Borealis Distributed Stream Processing Engine

GeoSensor Networks
An empirical study of high availability in stream processing systems

Proceedings of the 10th ACM/IFIP/USENIX International Conference on Middleware
Making every bit count in wide-area analytics

HotOS'13 Proceedings of the 14th USENIX conference on Hot Topics in Operating Systems
Aggregation and degradation in JetStream: streaming analytics in the wide area

NSDI'14 Proceedings of the 11th USENIX Conference on Networked Systems Design and Implementation

Quantified Score

Hi-index	0.00

Visualization

Abstract

We present a replication-based approach that enables both fast and reliable stream processing over wide area networks. Our approach replicates stream processing operators in a manner where operator replicas compete with each other to make the earliest impact. Therefore, any processing downstream from such replicas can proceed by relying on the fastest replica without being held back by slow or failed ones. Furthermore, our approach allows replicas to produce output in different orders so as to avoid the cost of forcing an identical execution across replicas, without sacrificing correctness. We first consider semantic issues for correct replicated stream processing and, based on a formal foundation, extend common stream-processing primitives. Next, we discuss strategies for deploying replicas. Finally, we present preliminary results obtained from experiments on Planet-Lab that substantiate the potential benefits of our approach.