Stream processing in data-driven computational science

  • Authors:

  • Ying Liu;Nithya Vijayakumar;Beth Plale

  • Affiliations:

  • Computer Science Department, Indiana University, Bloomington, IN, USA. yingliu@cs.indiana.edu;Computer Science Department, Indiana University, Bloomington, IN, USA. nvijayak@cs.indiana.edu;Computer Science Department, Indiana University, Bloomington, IN, USA. plale@cs.indiana.edu

  • Venue:
  • GRID '06 Proceedings of the 7th IEEE/ACM International Conference on Grid Computing
  • Year:
  • 2006

Quantified Score

Hi-index 0.00

Visualization

Abstract

The use of real-time data streams in data-driven computational science is driving the need for stream processing tools that work within the architectural framework of the larger application. Data stream processing systems are beginning to emerge in the commercial space, but these systems fail to address the needs of large-scale scientific applications. In this paper we illustrate the unique needs of large-scale data driven computational science through an example taken from weather prediction and forecasting. We apply a realistic workload from this application against our Calder stream processing system to determine effective throughput, event processing latency, data access scalability, and deployment latency.1