RIP: run-based intra-query parallelism for scalable complex event processing

• Authors: Cagri Balkesen;Nihal Dindar;Matthias Wetter;Nesime Tatbul
• Venue: Proceedings of the 7th ACM international conference on Distributed event-based systems
• Year: 2013

Recognition of patterns in event streams has become important in many application areas of Complex Event Processing (CEP) including financial markets, electronic health-care systems, and security monitoring systems. In most applications, patterns have to be detected continuously and in real-time over streams that are generated at very high rates, imposing high-performance requirements on the underlying CEP system. For scaling CEP systems to increasing workloads, parallel pattern matching techniques that can exploit multi-core processing opportunities are needed. In this paper, we propose RIP - a Run-based Intra-query Parallelism technique for scalable pattern matching over event streams. RIP distributes input events that belong to individual run instances of a pattern's Finite State Machine (FSM) to different processing units, thereby providing fine-grained partitioned data parallelism. We compare RIP to a state-based alternative which partitions individual FSM states to different processing units instead. Our experiments demonstrate that RIP's partitioned parallelism approach outperforms the pipelined parallelism approach of this state-based alternative, achieving near-linear scalability that is independent from the query pattern definition.