Stream-based event prediction using bayesian and bloom filters

  • Authors:
  • Miao Wang;Viliam Holub;John Murphy;Patrick O'Sullivan

  • Affiliations:
  • University College Dublin, Dublin, Ireland;University College Dublin, Dublin, Ireland;University College Dublin, Dublin, Ireland;IBM, Dublin, Ireland

  • Venue:
  • Proceedings of the 4th ACM/SPEC International Conference on Performance Engineering
  • Year:
  • 2013

Quantified Score

Hi-index 0.00

Visualization

Abstract

Nowadays, enterprise software systems store a large amount of operational information in logs. Manually analysing these data can be time-consuming and error-prone. Although a static knowledge database eases the task to capture recurring problems, maintaining such a knowledge repository requires periodic knowledge updates by domain experts. Moreover, as the repository grows, the problem of memory efficiency will also arise. Our goal is to enable administrators to efficiently capture interesting data in a high volume stream of events in real-time. We are proposing a statistical approach for software applications to be automatically trained with a smaller dataset to efficiently predict interesting data under such conditions. The proposed solution maintains a stable memory usage by migrating keywords from a dynamic data structure to fixed sized data structures (Bloom Filter). In particular, the solution has achieved better prediction results by enhancing the Bayesian theory with belief modifiers.