Detection accuracy of network anomalies using sampled flow statistics

Authors:
Ryoichi Kawahara;Keisuke Ishibashi;Tatsuya Mori;Noriaki Kamiyama;Shigeaki Harada;Haruhisa Hasegawa;Shoichiro Asano
Affiliations:
NTT Service Integration Laboratories, NTT Corporation, Tokyo, Japan;NTT Information Sharing Platform Laboratories, NTT Corporation, Tokyo, Japan;NTT Service Integration Laboratories, NTT Corporation, Tokyo, Japan;NTT Service Integration Laboratories, NTT Corporation, Tokyo, Japan;NTT-WEST Research and Development Center, Osaka, Japan;NTT Service Integration Laboratories, NTT Corporation, Tokyo, Japan;National Institute of Informatics, Tokyo, Japan
Venue:
International Journal of Network Management
Year:
2011

Citing 17
Cited 1

New directions in traffic measurement and accounting

Proceedings of the 2002 conference on Applications, technologies, architectures, and protocols for computer communications
A signal analysis of network traffic anomalies

Proceedings of the 2nd ACM SIGCOMM Workshop on Internet measurment
Properties and prediction of flow statistics from sampled packet streams

Proceedings of the 2nd ACM SIGCOMM Workshop on Internet measurment
Estimating flow distributions from sampled flow statistics

Proceedings of the 2003 conference on Applications, technologies, architectures, and protocols for computer communications
Sketch-based change detection: methods, evaluation, and applications

Proceedings of the 3rd ACM SIGCOMM conference on Internet measurement
Identifying elephant flows through periodically sampled packets

Proceedings of the 4th ACM SIGCOMM conference on Internet measurement
Traffic matrix estimation on a large IP backbone: a comparison on real data

Proceedings of the 4th ACM SIGCOMM conference on Internet measurement
Aberrant Behavior Detection in Time Series for Network Monitoring

LISA '00 Proceedings of the 14th USENIX conference on System administration
A robust system for accurate real-time summaries of internet traffic

SIGMETRICS '05 Proceedings of the 2005 ACM SIGMETRICS international conference on Measurement and modeling of computer systems
Mining anomalies using traffic feature distributions

Proceedings of the 2005 conference on Applications, technologies, architectures, and protocols for computer communications
Impact of packet sampling on anomaly detection metrics

Proceedings of the 6th ACM SIGCOMM conference on Internet measurement
Is sampled data sufficient for anomaly detection?

Proceedings of the 6th ACM SIGCOMM conference on Internet measurement
Combining filtering and statistical methods for anomaly detection

IMC '05 Proceedings of the 5th ACM SIGCOMM conference on Internet Measurement
A Study on Detecting Network Anomalies Using Sampled Flow Statistics

SAINT-W '07 Proceedings of the 2007 International Symposium on Applications and the Internet Workshops
Inferring Original Traffic Pattern from Sampled Flow Statistics

SAINT-W '07 Proceedings of the 2007 International Symposium on Applications and the Internet Workshops
Towards optimal sampling for flow size estimation

Proceedings of the 8th ACM SIGCOMM conference on Internet measurement
Fast monitoring of traffic subpopulations

Proceedings of the 8th ACM SIGCOMM conference on Internet measurement

Mean-variance relationship of the number of flows in traffic aggregation and its application to traffic management

Computer Networks: The International Journal of Computer and Telecommunications Networking

Quantified Score

Hi-index	0.00

Visualization

Abstract

We investigated the detection accuracy of network anomalies when using flow statistics obtained through packet sampling. Through a case study based on measurement data, we showed that network anomalies generating a large number of small flows, such as network scans or SYN flooding, become difficult to detect during packet sampling. We then developed an analytical model that enables us to quantitatively evaluate the effect of packet sampling and traffic conditions, such as anomalous traffic volume, on detection accuracy. We also investigated how the detection accuracy worsens when the packet sampling rate decreases. In addition, we show that, even with a low sampling rate, spatially partitioning monitored traffic into groups makes it possible to increase detection accuracy. We also developed a method of determining an appropriate number of partitioned groups, and we show its effectiveness. Copyright © 2011 John Wiley & Sons, Ltd.