Spatio-temporal network anomaly detection by assessing deviations of empirical measures

Authors:
Ioannis Ch. Paschalidis;Georgios Smaragdakis
Affiliations:
Center for Information and Systems Engineering, Department of Electrical and Computer Engineering, and Systems Engineering Division, Boston University, Brookline, MA;Deutsche Telekom Laboratories, Berlin, Germany
Venue:
IEEE/ACM Transactions on Networking (TON)
Year:
2009

Citing 20
Cited 4

A tutorial on hidden Markov models and selected applications in speech recognition

Readings in speech recognition
On the self-similar nature of Ethernet traffic (extended version)

IEEE/ACM Transactions on Networking (TON)
Self-similarity through high-variability: statistical analysis of Ethernet LAN traffic at the source level

IEEE/ACM Transactions on Networking (TON)
Self-similarity in World Wide Web traffic: evidence and possible causes

IEEE/ACM Transactions on Networking (TON)
Bro: a system for detecting network intruders in real-time

Computer Networks: The International Journal of Computer and Telecommunications Networking
The 1999 DARPA off-line intrusion detection evaluation

Computer Networks: The International Journal of Computer and Telecommunications Networking - Special issue on recent advances in intrusion detection systems
Model-based estimation of buffer overflow probabilities from measurements

Proceedings of the 2001 ACM SIGMETRICS international conference on Measurement and modeling of computer systems
A signal analysis of network traffic anomalies

Proceedings of the 2nd ACM SIGCOMM Workshop on Internet measurment
Information-Theoretic Measures for Anomaly Detection

SP '01 Proceedings of the 2001 IEEE Symposium on Security and Privacy
Sketch-based change detection: methods, evaluation, and applications

Proceedings of the 3rd ACM SIGCOMM conference on Internet measurement
A taxonomy of DDoS attack and DDoS defense mechanisms

ACM SIGCOMM Computer Communication Review
Diagnosing network-wide traffic anomalies

Proceedings of the 2004 conference on Applications, technologies, architectures, and protocols for computer communications
Characterization of network-wide anomalies in traffic flows

Proceedings of the 4th ACM SIGCOMM conference on Internet measurement
Snort - Lightweight Intrusion Detection for Networks

LISA '99 Proceedings of the 13th USENIX conference on System administration
Mining anomalies using traffic feature distributions

Proceedings of the 2005 conference on Applications, technologies, architectures, and protocols for computer communications
Network anomography

IMC '05 Proceedings of the 5th ACM SIGCOMM conference on Internet Measurement
An architecture for generating semantics-aware signatures

SSYM'05 Proceedings of the 14th conference on USENIX Security Symposium - Volume 14
On the stationarity of TCP bulk data transfers

PAM'05 Proceedings of the 6th international conference on Passive and Active Network Measurement
Wavelet analysis of long-range-dependent traffic

IEEE Transactions on Information Theory
On the estimation of buffer overflow probabilities from measurements

IEEE Transactions on Information Theory

Statistical anomaly detection with sensor networks

ACM Transactions on Sensor Networks (TOSN)
Global modeling of backbone network traffic

INFOCOM'10 Proceedings of the 29th conference on Information communications
An anomaly-based detection in ubiquitous network using the equilibrium state of the catastrophe theory

The Journal of Supercomputing
Anomaly secure detection methods by analyzing dynamic characteristics of the network traffic in cloud communications

Information Sciences: an International Journal

Quantified Score

Hi-index	0.00

Visualization

Abstract

We introduce an Internet traffic anomaly detection mechanism based on large deviations results for empirical measures. Using past traffic traces we characterize network traffic during various time-of-day intervals, assuming that it is anomaly-free. We present two different approaches to characterize traffic: (i) a model-free approach based on the method of types and Sanov's theorem, and (ii) a model-based approach modeling traffic using a Markov modulated process. Using these characterizations as a reference we continuously monitor traffic and employ large deviations and decision theory results to "compare" the empirical measure of the monitored traffic with the corresponding reference characterization, thus, identifying traffic anomalies in real-time. Our experimental results show that applying our methodology (even short-lived) anomalies are identified within a small number of observations. Throughout, we compare the two approaches presenting their advantages and disadvantages to identify and classify temporal network anomalies. We also demonstrate how our framework can be used to monitor traffic from multiple network elements in order to identify both spatial and temporal anomalies. We validate our techniques by analyzing real traffic traces with time-stamped anomalies.