Intrusion detection alarms reduction using root cause analysis and clustering

Authors:
Safaa O. Al-Mamory;Hongli Zhang
Affiliations:
School of Computer Science and Technology, Harbin Institute of Technology, 150001, China;School of Computer Science and Technology, Harbin Institute of Technology, 150001, China
Venue:
Computer Communications
Year:
2009

Citing 26
Cited 9

Silhouettes: a graphical aid to the interpretation and validation of cluster analysis

Journal of Computational and Applied Mathematics
A Validity Measure for Fuzzy Clustering

IEEE Transactions on Pattern Analysis and Machine Intelligence
Packets found on an internet

ACM SIGCOMM Computer Communication Review
Applied multivariate techniques

Applied multivariate techniques
Attribute-oriented induction in data mining

Advances in knowledge discovery and data mining
A new cluster validity index for the fuzzy c-mean

Pattern Recognition Letters
Data clustering: a review

ACM Computing Surveys (CSUR)
Data mining: concepts and techniques

Data mining: concepts and techniques
A data mining analysis of RTID alarms

Computer Networks: The International Journal of Computer and Telecommunications Networking - Special issue on recent advances in intrusion detection systems
Testing Intrusion detection systems: a critique of the 1998 and 1999 DARPA intrusion detection system evaluations as performed by Lincoln Laboratory

ACM Transactions on Information and System Security (TISSEC)
Principles of data mining

Principles of data mining
Data Mining Techniques: For Marketing, Sales, and Customer Support

Data Mining Techniques: For Marketing, Sales, and Customer Support
On Clustering Validation Techniques

Journal of Intelligent Information Systems
Data-Driven Discovery of Quantitative Rules in Relational Databases

IEEE Transactions on Knowledge and Data Engineering
Probabilistic Alert Correlation

RAID '00 Proceedings of the 4th International Symposium on Recent Advances in Intrusion Detection
Knowledge Discovery in Databases: An Attribute-Oriented Approach

VLDB '92 Proceedings of the 18th International Conference on Very Large Data Bases
Mining Alarm Clusters to Improve Alarm Handling Efficiency

ACSAC '01 Proceedings of the 17th Annual Computer Security Applications Conference
Clustering intrusion detection alarms to support root cause analysis

ACM Transactions on Information and System Security (TISSEC)
Techniques and tools for analyzing intrusion alerts

ACM Transactions on Information and System Security (TISSEC)
A Comprehensive Approach to Intrusion Detection Alert Correlation

IEEE Transactions on Dependable and Secure Computing
Snort - Lightweight Intrusion Detection for Networks

LISA '99 Proceedings of the 13th USENIX conference on System administration
Pattern Recognition, Fourth Edition

Pattern Recognition, Fourth Edition
Using unsupervised learning for network alert correlation

Canadian AI'08 Proceedings of the Canadian Society for computational studies of intelligence, 21st conference on Advances in artificial intelligence
A survey on IDS alerts processing techniques

ISP'07 Proceedings of the 6th WSEAS international conference on Information security and privacy
A Cluster Separation Measure

IEEE Transactions on Pattern Analysis and Machine Intelligence
Some new indexes of cluster validity

IEEE Transactions on Systems, Man, and Cybernetics, Part B: Cybernetics

Real-time classification of IDS alerts with data mining techniques

MILCOM'09 Proceedings of the 28th IEEE conference on Military communications
Mining negative generalized knowledge from relational databases

Knowledge-Based Systems
Alert correlation in collaborative intelligent intrusion detection systems-A survey

Applied Soft Computing
Towards a requirements-driven framework for detecting malicious behavior against software systems

Proceedings of the 2011 Conference of the Center for Advanced Studies on Collaborative Research
Requirements-Driven root cause analysis using markov logic networks

CAiSE'12 Proceedings of the 24th international conference on Advanced Information Systems Engineering
Inference of network anomaly propagation using spatio-temporal correlation

Journal of Network and Computer Applications
A comprehensive vulnerability based alert management approach for large networks

Future Generation Computer Systems
Network specific vulnerability based alert reduction approach

Security and Communication Networks
Perceived causes of software project failures - An analysis of their relationships

Information and Software Technology

Quantified Score

Hi-index	0.24

Visualization

Abstract

As soon as the Intrusion Detection System (IDS) detects any suspicious activity, it will generate several alarms referring to as security breaches. Unfortunately, the triggered alarms usually are accompanied with huge number of false positives. In this paper, we use root cause analysis to discover the root causes making the IDS triggers these false alarms; most of these root causes are not attacks. Removing the root causes enhances alarms quality in the future. The root cause instigates the IDS to trigger alarms that almost always have similar features. These similar alarms can be clustered together; consequently, we have designed a new clustering technique to group IDS alarms and to produce clusters. Then, each cluster is modeled by a generalized alarm. The generalized alarms related to root causes are converted (by the security analyst) to filters in order to reduce future alarms' load. The suggested system is a semi-automated system helping the security analyst in specifying the root causes behind these false alarms and in writing accurate filtering rules. The proposed clustering method was verified with three different datasets, and the averaged reduction ratio was about 74% of the total alarms. Application of the new technique to alarms log greatly helps the security analyst in identifying the root causes; and then reduces the alarm load in the future.