Processing of flow accounting data in Java: framework design and performance evaluation

Authors:
Jochen Kögel;Sebastian Scholz
Affiliations:
Institute of Communication Networks and Computer Engineering, University of Stuttgart, Stuttgart;Institute of Communication Networks and Computer Engineering, University of Stuttgart, Stuttgart
Venue:
EUNICE'10 Proceedings of the 16th EUNICE/IFIP WG 6.6 conference on Networked services and applications: engineering, control and management
Year:
2010

Citing 3
Cited 0

Gigascope: a stream database for network applications

Proceedings of the 2003 ACM SIGMOD international conference on Management of data
A Framework for Real-Time Worm Attack Detection and Backbone Monitoring

IWCIP '05 Proceedings of the First IEEE International Workshop on Critical Infrastructure Protection
Design of a Stream-Based IP Flow Record Query Language

DSOM '09 Proceedings of the 20th IFIP/IEEE International Workshop on Distributed Systems: Operations and Management: Integrated Management of Systems, Services, Processes and People in IT

Quantified Score

Hi-index	0.00

Visualization

Abstract

Flow Accounting is a passive monitoring mechanism implemented in routers that gives insight into traffic behavior and network characteristics. However, processing of Flow Accounting data is a challenging task, especially in large networks where the rate of Flow Records received at the collector can be very high. We developed a framework for processing of Flow Accounting data in Java. It provides processing blocks for aggregation, sorting, statistics, correlation, and other tasks. Besides reading data from files for offline analysis, it can also directly process data received from the network. In terms of multithreading and data handling, the framework is highly configurable, which allows performance tuning depending on the given task. For setting these parameters there are several trade-offs concerning memory consumption and processing overhead. In this paper, we present the framework design, study these trade-offs based on a reference scenario and examine characteristics caused by garbage collection.