A novel unsupervised classification approach for network anomaly detection by k-Means clustering and ID3 decision tree learning methods

Authors:
Yasser Yasami;Saadat Pour Mozaffari
Affiliations:
Computer Engineering Department, Amirkabir University of Technology (AUT), Tehran, Iran;Computer Engineering Department, Amirkabir University of Technology (AUT), Tehran, Iran
Venue:
The Journal of Supercomputing
Year:
2010

Citing 17
Cited 1

On Combining Classifiers

IEEE Transactions on Pattern Analysis and Machine Intelligence
Soft combination of neural classifiers: a comparative study

Pattern Recognition Letters
Machine Learning

Machine Learning
Multivariate Statistical Analysis of Audit Trails for Host-Based Intrusion Detection

IEEE Transactions on Computers
KDD-99 classifier learning contest LLSoft's results overview

ACM SIGKDD Explorations Newsletter
Anomaly detection of web-based attacks

Proceedings of the 10th ACM conference on Computer and communications security
Pattern Classification (2nd Edition)

Pattern Classification (2nd Edition)
Symbolic dynamic analysis of complex systems for anomaly detection

Signal Processing
A New Dependency and Correlation Analysis for Features

IEEE Transactions on Knowledge and Data Engineering
Controlling the effects of anomalous ARP behaviour on ethernet networks

CoNEXT '05 Proceedings of the 2005 ACM conference on Emerging network experiment and technology
Anomalous system call detection

ACM Transactions on Information and System Security (TISSEC)
A Multivariate Adaptive Method for Detecting ARP Anomaly in Local Area Networks

ICSNC '06 Proceedings of the International Conference on Systems and Networks Communication
K-Means+ID3: A Novel Method for Supervised Anomaly Detection by Cascading K-Means Clustering and ID3 Decision Tree Learning Methods

IEEE Transactions on Knowledge and Data Engineering
An ARP-based Anomaly Detection Algorithm Using Hidden Markov Model in Enterprise Networks

ICSNC '07 Proceedings of the Second International Conference on Systems and Networks Communications
Anomaly detection in IP networks

IEEE Transactions on Signal Processing
Switching between selection and fusion in combining classifiers: anexperiment

IEEE Transactions on Systems, Man, and Cybernetics, Part B: Cybernetics
Hierarchical Kohonenen net for anomaly detection in network security

IEEE Transactions on Systems, Man, and Cybernetics, Part B: Cybernetics

Effects-based feature identification for network intrusion detection

Neurocomputing

Quantified Score

Hi-index	0.00

Visualization

Abstract

This paper presents a novel host-based combinatorial method based on k-Means clustering and ID3 decision tree learning algorithms for unsupervised classification of anomalous and normal activities in computer network ARP traffic. The k-Means clustering method is first applied to the normal training instances to partition it into k clusters using Euclidean distance similarity. An ID3 decision tree is constructed on each cluster. Anomaly scores from the k-Means clustering algorithm and decisions of the ID3 decision trees are extracted. A special algorithm is used to combine results of the two algorithms and obtain final anomaly score values. The threshold rule is applied for making the decision on the test instance normality. Experiments are performed on captured network ARP traffic. Some anomaly criteria has been defined and applied to the captured ARP traffic to generate normal training instances. Performance of the proposed approach is evaluated using five defined measures and empirically compared with the performance of individual k-Means clustering and ID3 decision tree classification algorithms and the other proposed approaches based on Markovian chains and stochastic learning automata. Experimental results show that the proposed approach has specificity and positive predictive value of as high as 96 and 98%, respectively.