Resource allocation for multiple concurrent in-network stream-processing applications

  • Authors:
  • Anne Benoit;Henri Casanova;Veronika Rehn-Sonigo;Yves Robert

  • Affiliations:
  • LIP, ENS Lyon, 46 allée d'Italie, 69364 Lyon Cedex 07, France;University of Hawai'i at Manoa, 1680 East-West Road, Honolulu, HI 96822, USA;LIG, ENSIMAG, 51, avenue Jean Kuntzmann, 38330 Montbonnot Saint Martin, France;LIP, ENS Lyon, 46 allée d'Italie, 69364 Lyon Cedex 07, France

  • Venue:
  • Parallel Computing
  • Year:
  • 2011

Quantified Score

Hi-index 0.00

Visualization

Abstract

This work investigates the operator mapping problem for in-network stream-processing. In a stream-processing application, a tree of operators is applied, in steady-state mode, to datasets that are continuously updated at different locations in the network. The goal is to generate updated final results at a desired rate. In in-network stream-processing, dataset updates and operator computations are performed by servers distributed in a network. We consider the problem of mapping operators to these servers in the case of multiple concurrent stream-processing applications. In this case, different operator trees corresponding to different applications may share common subtrees, so that intermediate results can be reused by different applications. This work provides complexity results for different versions of the operator mapping problem, which can be formulated as integer linear programs. Several polynomial-time heuristics are proposed for a particularly relevant version of the problem, which is NP-hard. These heuristics are compared and evaluated via simulation. The results demonstrate the importance of mapping the operators to appropriate processors, and the importance of sharing common sub-trees across operator trees.