Event Stream Processing with Out-of-Order Data Arrival

  • Authors:
  • Ming Li;Mo Liu;Luping Ding;Elke A. Rundensteiner;Murali Mani

  • Affiliations:
  • Worcester Polytechnic Institute, USA;Worcester Polytechnic Institute, USA;Worcester Polytechnic Institute, USA;Worcester Polytechnic Institute, USA;Worcester Polytechnic Institute, USA

  • Venue:
  • ICDCSW '07 Proceedings of the 27th International Conference on Distributed Computing Systems Workshops
  • Year:
  • 2007

Quantified Score

Hi-index 0.00

Visualization

Abstract

Complex event processing has become increasingly important in modern applications, ranging from supply chain management for RFID tracking to real-time intrusion detection. The goal is to extract patterns from such event streams in order to make informed decisions in real-time. However, networking latencies and even machine failure may cause events to arrive out-of-order at the event stream processing engine. In this work, we address the problem of processing event pattern queries specified over event streams that may contain out-of-order data. First, we analyze the problems state-of-the-art event stream processing technology would experience when faced with out-of-order data arrival. We then propose a new solution of physical implementation strategies for the core stream algebra operators such as sequence scan and pattern construction, including stack-based data structures and associated purge algorithms. Optimizations for sequence scan and construction as well as state purging to minimize CPU cost and memory consumption are also introduced. Lastly, we conduct an experimental study demonstrating the effectiveness of our approach.