The MeDICi Integration Framework: A Platform for High Performance Data Streaming Applications

  • Authors:
  • Ian Gorton;Adam Wynne;Justin Almquist;Jack Chatterton

  • Affiliations:
  • -;-;-;-

  • Venue:
  • WICSA '08 Proceedings of the Seventh Working IEEE/IFIP Conference on Software Architecture (WICSA 2008)
  • Year:
  • 2008

Quantified Score

Hi-index 0.00

Visualization

Abstract

Building high performance analytical applications for data streams generated from sensors is a challenging software engineering problem. Such applications typically comprise a complex pipeline of processing components that capture, transform and analyze the incoming data stream. In addition, applications must provide high throughput, be scalable and easily modifiable so that new analytical components can be added with minimum effort. In this paper we describe the MeDICi Integration Framework (MIF), which is a middleware platform we have created to address these challenges. The MIF extends an open source messaging platform with a component-based API for integrating components into analytical pipelines. We describe the features and capabilities of the MIF, and show how it has been used to build a production analytical application for detecting cyber security attacks. The application was composed from multiple independently developed components using several different programming languages. The resulting application was able to process network sensor traffic in real time and provide insightful feedback to network analysts as soon as potential attacks were recognized.