Processing High-Volume Stream Queries on a Supercomputer

  • Authors:
  • Erik Zeitler;Tore Risch

  • Affiliations:
  • Uppsala University;Uppsala University

  • Venue:
  • ICDEW '06 Proceedings of the 22nd International Conference on Data Engineering Workshops
  • Year:
  • 2006

Quantified Score

Hi-index 0.00

Visualization

Abstract

Scientific instruments, such as radio telescopes, colliders, sensor networks, and simulators generate very high volumes of data streams that scientists analyze to detect and understand physical phenomena. The high data volume and the need for advanced computations on the streams require substantial hardware resources and scalable stream processing. We address these challenges by developing data stream management technology to support high-volume stream queries utilizing massively parallel computer hardware. We have developed a data stream management system prototype for state-of-the-art parallel hardware. The performance evaluation uses real measurement data from LOFAR, a radio telescope antenna array being developed in the Netherlands.