From a stream of relational queries to distributed stream processing

  • Authors:
  • Qiong Zou;Huayong Wang;Robert Soulé;Martin Hirzel;Henrique Andrade;Buǧra Gedik;Kun-Lung Wu

  • Affiliations:
  • IBM Corporation;IBM Corporation;New York University;T. J. Watson Research Center;T. J. Watson Research Center;T. J. Watson Research Center;T. J. Watson Research Center

  • Venue:
  • Proceedings of the VLDB Endowment
  • Year:
  • 2010

Quantified Score

Hi-index 0.00

Visualization

Abstract

Applications from several domains are now being written to process live data originating from hardware and software-based streaming sources. Many of these applications have been written relying solely on database and data warehouse technologies, despite their lack of need for transactional support and ACID properties. In several extreme high-load cases, this approach does not scale to the processing speeds that these applications demand. In this paper we demonstrate an application acceleration approach whereby a regular ODBC-based application is converted into a true streaming application with minimal disruption from a software engineering standpoint. We showcase our approach on three real-world applications. We experimentally demonstrate the substantial performance improvements that can be observed when contrasting the accelerated implementation with the original database-oriented implementation.