Performance evaluation of object-oriented active database systems using the BEAST benchmark
Theory and Practice of Object Systems
Linear road: a stream data management benchmark
VLDB '04 Proceedings of the Thirtieth international conference on Very large data bases - Volume 30
Benchmarking event processing systems: current state and future directions
Proceedings of the first joint WOSP/SIPEW international conference on Performance engineering
The interaction ontology: low-level cue processing in real-time group conversations
Proceedings of the 2nd ACM international workshop on Events in multimedia
A logistics workload for event notification middleware
From active data management to event-based systems and more
Pattern rewriting framework for event processing optimization
Proceedings of the 5th ACM international conference on Distributed event-based system
Towards vulnerability-based intrusion detection with event processing
Proceedings of the 5th ACM international conference on Distributed event-based system
Proceedings of the 5th ACM international conference on Distributed event-based system
Proceedings of the 6th ACM International Conference on Distributed Event-Based Systems
FINCoS: benchmark tools for event processing systems
Proceedings of the 4th ACM/SPEC International Conference on Performance Engineering
Hi-index | 0.00 |
Several new Complex Event Processing (CEP) engines have been recently released, many of which are intended to be used in performance sensitive scenarios - like fraud detection, traffic control, or health care systems. However, there is no standard means to assess the performance of a CEP engine. This omission is all the more relevant as there are currently many competing products, languages, architectures, data models, and data processing CEP techniques. A performance evaluation framework can help identify good design decisions and assist in improving engines. Here we demonstrate our work in progress: FINCoS, a framework that can be used to benchmark CEP systems. The proposed framework has five relevant characteristics: i. Flexible (e.g., it allows changing the workload on the fly to measure reactions to peak loads); ii. Independent of particular workloads; iii. Neutral (not bound to any specific CEP product); iv. Correctness check (validators can be plugged into the framework on demand to verify results); v. Scalable (many of its components, like event generators, can be centrally orchestrated and run in parallel). Note that the framework does not include a benchmark specification. In fact, it was designed such that diverse datasets and query scenarios can be easily attached and tested on several CEP engines. As such, this framework has three key benefits: first, it can be used by the CEP community to more quickly devise and experiment new benchmarks for event processing systems. Second, CEP vendors can employ the framework in conjunction with their own test datasets to benchmark their systems internally. Finally, customers can use it with their real data and select the CEP engine that best fits their needs.