Dynamic expressivity with static optimization for streaming languages

  • Authors:

  • Robert Soulé;Michael I. Gordon;Saman Amarasinghe;Robert Grimm;Martin Hirzel

  • Affiliations:

  • Cornell University, Ithaca, NY, USA;Massachusetts Institute of Technology, Cambridge, MA, USA;Massachusetts Institute of Technology, Cambridge, MA, USA;New York University, New York City, NY, USA;IBM Research, Yorktown Heights, NY, USA

  • Venue:
  • Proceedings of the 7th ACM international conference on Distributed event-based systems
  • Year:
  • 2013

Quantified Score

Hi-index 0.00

Visualization

Abstract

Developers increasingly use streaming languages to write applications that process large volumes of data with high throughput. Unfortunately, when picking which streaming language to use, they face a difficult choice. On the one hand, dynamically scheduled languages allow developers to write a wider range of applications, but cannot take advantage of many crucial optimizations. On the other hand, statically scheduled languages are extremely performant, but have difficulty expressing many important streaming applications. This paper presents the design of a hybrid scheduler for stream processing languages. The compiler partitions the streaming application into coarse-grained subgraphs separated by dynamic rate boundaries. It then applies static optimizations to those subgraphs. We have implemented this scheduler as an extension to the StreamIt compiler. To evaluate its performance, we compare it to three scheduling techniques used by dynamic systems (OS thread, demand, and no-op) on a combination of micro-benchmarks and real-world inspired synthetic benchmarks. Our scheduler not only allows the previously static version of StreamIt to run dynamic rate applications, but it outperforms the three dynamic alternatives. This demonstrates that our scheduler strikes the right balance between expressivity and performance for stream processing languages.