Virtualizing stream processing

  • Authors:
  • Michael Duller;Jan S. Rellermeyer;Gustavo Alonso;Nesime Tatbul

  • Affiliations:
  • Systems Group, Department of Computer Science, ETH Zurich, Zurich, Switzerland;IBM Austin Research Laboratory, Austin, TX;Systems Group, Department of Computer Science, ETH Zurich, Zurich, Switzerland;Systems Group, Department of Computer Science, ETH Zurich, Zurich, Switzerland

  • Venue:
  • Middleware'11 Proceedings of the 12th ACM/IFIP/USENIX international conference on Middleware
  • Year:
  • 2011

Quantified Score

Hi-index 0.00

Visualization

Abstract

Stream processing systems have evolved into established solutions as standalone engines but they still lack flexibility in terms of large-scale deployment, integration, extensibility, and interoperability. In the last years, a substantial ecosystem of new applications has emerged that can potentially benefit from stream processing but introduces different requirements on how stream processing solutions can be integrated, deployed, extended, and federated. To address these needs, we present an exoengine architecture and the associated ExoP platform. Together, they provide the means for encapsulating components of stream processing systems as well as automating the data exchange between components and their distributed deployment. The proposed solution can be used, e.g., to connect heterogeneous streaming engines, replace operators at runtime, and migrate operators across machines with a negligible overhead.