Extracting hidden anomalies using sketch and non Gaussian multiresolution statistical detection procedures

Authors:
Guillaume Dewaele;Kensuke Fukuda;Pierre Borgnat;Patrice Abry;Kenjiro Cho
Affiliations:
Physics Laboratory, France;National Institute of Informatics, Tokyo, Japan;Physics Laboratory, France;Physics Laboratory, France;Internet Initiative Japan, Tokyo, Japan
Venue:
Proceedings of the 2007 workshop on Large scale attack defense
Year:
2007

Citing 21
Cited 35

Distance measures for signal processing and pattern recognition

Signal Processing
A non-instrusive, wavelet-based approach to detecting network performance problems

IMW '01 Proceedings of the 1st ACM SIGCOMM Workshop on Internet Measurement
Flash crowds and denial of service attacks: characterization and implications for CDNs and web sites

Proceedings of the 11th international conference on World Wide Web
Self-Similar Network Traffic and Performance Evaluation

Self-Similar Network Traffic and Performance Evaluation
A signal analysis of network traffic anomalies

Proceedings of the 2nd ACM SIGCOMM Workshop on Internet measurment
Data streams: algorithms and applications

SODA '03 Proceedings of the fourteenth annual ACM-SIAM symposium on Discrete algorithms
A framework for classifying denial of service attacks

Proceedings of the 2003 conference on Applications, technologies, architectures, and protocols for computer communications
Sketch-based change detection: methods, evaluation, and applications

Proceedings of the 3rd ACM SIGCOMM conference on Internet measurement
Tabulation based 4-universal hashing with applications to second moment estimation

SODA '04 Proceedings of the fifteenth annual ACM-SIAM symposium on Discrete algorithms
A taxonomy of DDoS attack and DDoS defense mechanisms

ACM SIGCOMM Computer Communication Review
Diagnosing network-wide traffic anomalies

Proceedings of the 2004 conference on Applications, technologies, architectures, and protocols for computer communications
Aberrant Behavior Detection in Time Series for Network Monitoring

LISA '00 Proceedings of the 14th USENIX conference on System administration
Manifold learning visualization of network traffic data

Proceedings of the 2005 ACM SIGCOMM workshop on Mining network data
PacketScore: A Statistics-Based Packet Filtering Scheme against Distributed Denial-of-Service Attacks

IEEE Transactions on Dependable and Secure Computing
Detection and identification of network anomalies using sketch subspaces

Proceedings of the 6th ACM SIGCOMM conference on Internet measurement
Network anomography

IMC '05 Proceedings of the 5th ACM SIGCOMM conference on Internet Measurement
Combining filtering and statistical methods for anomaly detection

IMC '05 Proceedings of the 5th ACM SIGCOMM conference on Internet Measurement
Inferring internet denial-of-service activity

SSYM'01 Proceedings of the 10th conference on USENIX Security Symposium - Volume 10
Non-Gaussian and Long Memory Statistical Characterizations for Internet Traffic with Anomalies

IEEE Transactions on Dependable and Secure Computing
Invited Talk: Sketch Based Anomaly Detection, Identification and Performance Evaluation

SAINT-W '07 Proceedings of the 2007 International Symposium on Applications and the Internet Workshops
Traffic data repository at the WIDE project

ATEC '00 Proceedings of the annual conference on USENIX Annual Technical Conference

Continuous Time Bayesian Networks for Host Level Network Intrusion Detection

ECML PKDD '08 Proceedings of the European conference on Machine Learning and Knowledge Discovery in Databases - Part II
An image processing approach to traffic anomaly detection

Proceedings of the 4th Asian Conference on Internet Engineering
Anomaly extraction in backbone networks using association rules

Proceedings of the 9th ACM SIGCOMM conference on Internet measurement conference
Towards systematic traffic annotation

Proceedings of the 5th international student workshop on Emerging networking experiments and technologies
A visualization tool for exploring multi-scale network traffic anomalies

SPECTS'09 Proceedings of the 12th international conference on Symposium on Performance Evaluation of Computer & Telecommunication Systems
Combining multiresolution and random projections for robust statistical analysis and anomaly detection

AINTEC '09 Asian Internet Engineering Conference
Anomaly detection in IP networks with principal component analysis

ISCIT'09 Proceedings of the 9th international conference on Communications and information technologies
On the use of sketches and wavelet analysis for network anomaly detection

Proceedings of the 6th International Wireless Communications and Mobile Computing Conference
An automatic and dynamic parameter tuning of a statistic-based anomaly detection algorithm

ICC'09 Proceedings of the 2009 IEEE international conference on Communications
Distribution-based anomaly detection in 3G mobile networks: from theory to practice

International Journal of Network Management
A scalable, efficient and informative approach for anomaly-based intrusion detection systems: theory and practice

International Journal of Network Management
An evaluation of automatic parameter tuning of a statistics-based anomaly detection algorithm

International Journal of Network Management
Unsupervised host behavior classification from connection patterns

International Journal of Network Management
MAWILab: combining diverse anomaly detectors for automated anomaly labeling and performance benchmarking

Proceedings of the 6th International COnference
Intrusion detection using continuous time Bayesian networks

Journal of Artificial Intelligence Research
A Hough-transform-based anomaly detector with an adaptive time interval

Proceedings of the 2011 ACM Symposium on Applied Computing
sub-space clustering and evidence accumulation for unsupervised network anomaly detection

TMA'11 Proceedings of the Third international conference on Traffic monitoring and analysis
An analysis of longitudinal TCP passive measurements

TMA'11 Proceedings of the Third international conference on Traffic monitoring and analysis
UNADA: unsupervised network anomaly detection using sub-space outliers ranking

NETWORKING'11 Proceedings of the 10th international IFIP TC 6 conference on Networking - Volume Part I
Accurate network anomaly classification with generalized entropy metrics

Computer Networks: The International Journal of Computer and Telecommunications Networking
A Hough-transform-based anomaly detector with an adaptive time interval

ACM SIGAPP Applied Computing Review
Traffic causality graphs: profiling network applications through temporal and spatial causality of flows

Proceedings of the 23rd International Teletraffic Congress
Uncovering relations between traffic classifiers and anomaly detectors via graph theory

TMA'10 Proceedings of the Second international conference on Traffic Monitoring and Analysis
Sub-space clustering, inter-clustering results association & anomaly correlation for unsupervised network anomaly detection

Proceedings of the 7th International Conference on Network and Services Management
0day anomaly detection made possible thanks to machine learning

WWIC'10 Proceedings of the 8th international conference on Wired/Wireless Internet Communications
A longitudinal study of small-time scaling behavior of internet traffic

NETWORKING'10 Proceedings of the 9th IFIP TC 6 international conference on Networking
Unsupervised Network Intrusion Detection Systems: Detecting the Unknown without Knowledge

Computer Communications
On the recent use of email through traffic and network analysis: the impact of OSNs, new trends, and other communication platforms

ACM SIGMETRICS Performance Evaluation Review
Improving an SVD-based combination strategy of anomaly detectors for traffic labelling

Proceedings of the Asian Internet Engineeering Conference
Disclosure: detecting botnet command and control servers through large-scale NetFlow analysis

Proceedings of the 28th Annual Computer Security Applications Conference
Anomaly extraction in backbone networks using association rules

IEEE/ACM Transactions on Networking (TON)
ADMIRE: Anomaly detection method using entropy-based PCA with three-step sketches

Computer Communications
Beehive: large-scale log analysis for detecting suspicious activity in enterprise networks

Proceedings of the 29th Annual Computer Security Applications Conference
Synoptic graphlet: bridging the gap between supervised and unsupervised profiling of host-level network traffic

IEEE/ACM Transactions on Networking (TON)
A methodological overview on anomaly detection

DataTraffic Monitoring and Analysis

Quantified Score

Hi-index	0.00

Visualization

Abstract

A new profile-based anomaly detection and characterization procedure is proposed. It aims at performing prompt and accurate detection of both short-lived and long-lasting low-intensity anomalies, without the recourse of any prior knowledge of the targetted traffic. Key features of the algorithm lie in the joint use of random projection techniques (sketches) and of a multiresolution non Gaussian marginal distribution modeling. The former enables both a reduction in the dimensionality of the data and the measurement of the reference (i.e., normal) traffic behavior, while the latter extracts anomalies at different aggregation levels. This procedure is used to blindly analyze a large-scale packet trace database collected on a trans-Pacific transit link from 2001 to 2006. It can detect and identify a large number of known and unknown anomalies and attacks, whose intensities are low (down to below one percent). Using sketches also makes possible a real-time identification of the source or destination IP addresses associated to the detected anomaly and hence their mitigation.