ARGUS: rete + DBMS = efficient persistent profile matching on large-volume data streams

  • Authors:
  • Chun Jin;Jaime Carbonell;Phil Hayes

  • Affiliations:
  • Language Technologies Institute, School of Computer Science, Carnegie Mellon University, Pittsburgh, PA;Language Technologies Institute, School of Computer Science, Carnegie Mellon University, Pittsburgh, PA;Dynamix Technologies, Wexford, PA

  • Venue:
  • ISMIS'05 Proceedings of the 15th international conference on Foundations of Intelligent Systems
  • Year:
  • 2005

Quantified Score

Hi-index 0.00

Visualization

Abstract

Efficient processing of complex streaming data presents multiple challenges, especially when combined with intelligent detection of hidden anomalies in real time. We label such systems Stream Anomaly Monitoring Systems (SAMS), and describe the CMU/Dynamix ARGUS system as a new kind of SAMS to detect rare but high value patterns combining streaming and historical data. Such patterns may correspond to hidden precursors of terrorist activity, or early indicators of the onset of a dangerous disease, such as a SARS outbreak. Our method starts from an extension of the RETE algorithm for matching streaming data against multiple complex persistent queries, and proceeds beyond to transitivity inferences, conditional intermediate result materialization, and other such techniques to obtain both accuracy and efficiency, as demonstrated by the evaluation results outperforming classical techniques such as a modern DMBS.