Design principles for developing stream processing applications

  • Authors:

  • Deepak Turaga;Henrique Andrade;Buğra Gedik;Chitra Venkatramani;Olivier Verscheure;Jeffrey David Harris;John Cox;William Szewczyk;Paul Jones

  • Affiliations:

  • IBM Research, 19 Skyline Dr, Hawthorne, NY 10532, U.S.A.;IBM Research, 19 Skyline Dr, Hawthorne, NY 10532, U.S.A.;IBM Research, 19 Skyline Dr, Hawthorne, NY 10532, U.S.A.;IBM Research, 19 Skyline Dr, Hawthorne, NY 10532, U.S.A.;IBM Research, 19 Skyline Dr, Hawthorne, NY 10532, U.S.A.;U.S. Government, Washington, District of Columbia, U.S.A.;U.S. Government, Washington, District of Columbia, U.S.A.;U.S. Government, Washington, District of Columbia, U.S.A.;HM Government, London, U.K.

  • Venue:
  • Software—Practice & Experience - Focus on Selected PhD Literature Reviews in the Practical Aspects of Software Technology
  • Year:
  • 2010

Quantified Score

Hi-index 0.00

Visualization

Abstract

Stream processing applications are used to ingest, process, and analyze continuous data streams from heterogeneous sources of live and stored data, generating streams of output results. These applications are, in many cases, complex, large-scale, low-latency, and distributed in nature. In this paper, we describe the design principles and architectural underpinnings for stream processing applications. These principles are distilled from our experience in building real-world applications both for internal use as well as with customers from several industrial and academic domains. We provide principles, guidelines, as well as appropriate implementation examples to highlight the different aspects of stream processing application design and development. Copyright © 2010 John Wiley & Sons, Ltd.