Agile and scalable analysis of network events

  • Authors:
  • Mike Fisk;George Varghese

  • Affiliations:
  • Los Alamos National Laboratory and the University of California San Diego;The University of California San Diego

  • Venue:
  • Proceedings of the 2nd ACM SIGCOMM Workshop on Internet measurment
  • Year:
  • 2002

Quantified Score

Hi-index 0.00

Visualization

Abstract

The state of the art in general purpose software systems for large-scale traffic measurement has not progressed much past the venerable libpcap. In this paper we describe a new data analysis system that provides a scalable, flexible system for composing ad-hoc analyses of high-speed, streming data. This agility allows researchers, network security analysts, or network operators to easily compose new analysis functions. A growing tool box of filtering, measurement, and statistical tools allows new approaches to be tested with a minimum of software development. Further, a dynamic type system allows polymorphic analysis modules to operate on arbitrary forms of structured data, thus allowing easy integration of multiple data sources such as packet traces, netflow records, or security logs. In this paper we present this system and demonstrate its capabilities while performing several measurements, such as computing probability density functions, detecting port-scans, and probabilistic counting of traffic traces.