Scalable complex event processing on top of mapreduce

Authors:
Jiaxue Yang;Yu Gu;Yubin Bao;Ge Yu
Affiliations:
Northeastern University, China;Northeastern University, China;Northeastern University, China;Northeastern University, China
Venue:
APWeb'12 Proceedings of the 14th Asia-Pacific international conference on Web Technologies and Applications
Year:
2012

Citing 6
Cited 0

High-performance complex event processing over streams

Proceedings of the 2006 ACM SIGMOD international conference on Management of data
Distributed complex event processing with query rewriting

Proceedings of the Third ACM International Conference on Distributed Event-Based Systems
MapReduce: a flexible data processing tool

Communications of the ACM - Amir Pnueli: Ahead of His Time
Hadoop: The Definitive Guide

Hadoop: The Definitive Guide
Changing flights in mid-air: a model for safely modifying continuous queries

Proceedings of the 2011 ACM SIGMOD International Conference on Management of data
TI: an efficient indexing mechanism for real-time search on tweets

Proceedings of the 2011 ACM SIGMOD International Conference on Management of data

Quantified Score

Hi-index	0.00

Visualization

Abstract

In this paper, we propose a complex event processing framework on top of MapReduce, which may be widely used in many fields, such as the RFID monitoring and tracking, the intrusion detection and so on. In our framework, data collectors collect events and upload them to distributed file systems asynchronously. Then the MapReduce programming model is utilized to detect and identify events in parallel. Meanwhile, our framework also supports continuous queries over event streams by the cache mechanism. In order to reduce the delay of detecting and processing events, we replace the merge-sort phase in MapReduce tasks with hybrid sort. Also, the results can be responded in the real-time manner to users using the feedback mechanism. The feasibility and efficiency of our proposed framework are verified by the experiments.