IEEE Transactions on Software Engineering - Special issue on computer security and privacy
Elements of information theory
Elements of information theory
A Methodology for Testing Intrusion Detection Systems
IEEE Transactions on Software Engineering
The base-rate fallacy and its implications for the difficulty of intrusion detection
CCS '99 Proceedings of the 6th ACM conference on Computer and communications security
Bro: a system for detecting network intruders in real-time
Computer Networks: The International Journal of Computer and Telecommunications Networking
ACM Transactions on Information and System Security (TISSEC)
Statistical Foundations of Audit Trail Analysis for the Detection of Computer Misuse
IEEE Transactions on Software Engineering
Benchmarking Anomaly-Based Detection Systems
DSN '00 Proceedings of the 2000 International Conference on Dependable Systems and Networks (formerly FTCS-30 and DCCA-8)
Information-Theoretic Measures for Anomaly Detection
SP '01 Proceedings of the 2001 IEEE Symposium on Security and Privacy
Evaluation of Intrusion Detectors: A Decision Theory Approach
SP '01 Proceedings of the 2001 IEEE Symposium on Security and Privacy
Relationship-based clustering and cluster ensembles for high-dimensional data mining
Relationship-based clustering and cluster ensembles for high-dimensional data mining
Snort - Lightweight Intrusion Detection for Networks
LISA '99 Proceedings of the 13th USENIX conference on System administration
Application layer intrusion detection for SQL injection
Proceedings of the 44th annual Southeast regional conference
Principled reasoning and practical applications of alert fusion in intrusion detection systems
Proceedings of the 2008 ACM symposium on Information, computer and communications security
RAID '08 Proceedings of the 11th international symposium on Recent Advances in Intrusion Detection
ACM Transactions on Autonomous and Adaptive Systems (TAAS)
A detailed analysis of the KDD CUP 99 data set
CISDA'09 Proceedings of the Second IEEE international conference on Computational intelligence for security and defense applications
Security alert correlation using growing neural gas
CISIS'11 Proceedings of the 4th international conference on Computational intelligence in security for information systems
Information-theoretic modeling of false data filtering schemes in wireless sensor networks
ACM Transactions on Sensor Networks (TOSN)
Towards an information-theoretic framework for analyzing intrusion detection systems
ESORICS'06 Proceedings of the 11th European conference on Research in Computer Security
Cooperative intrusion detection for web applications
CANS'06 Proceedings of the 5th international conference on Cryptology and Network Security
A self-tuning self-optimizing approach for automated network anomaly detection systems
Proceedings of the 9th international conference on Autonomic computing
Botnets: a heuristic-based detection framework
Proceedings of the Fifth International Conference on Security of Information and Networks
Evaluation of classification algorithms for intrusion detection in MANETs
Knowledge-Based Systems
Performance analysis of wireless intrusion detection systems
IDCS'12 Proceedings of the 5th international conference on Internet and Distributed Computing Systems
Divided two-part adaptive intrusion detection system
Wireless Networks
Administrative evaluation of intrusion detection system
Proceedings of the 2nd annual conference on Research in information technology
Engineering Applications of Artificial Intelligence
Towards cost-sensitive assessment of intrusion response selection
Journal of Computer Security
Hi-index | 0.00 |
A fundamental problem in intrusion detection is what metric(s) can be used to objectively evaluate an intrusion detection system (IDS) in terms of its ability to correctly classify events as normal or intrusive. Traditional metrics (e.g., true positive rate and false positive rate) measure different aspects, but no single metric seems sufficient to measure the capability of intrusion detection systems. The lack of a single unified metric makes it difficult to fine-tune and evaluate an IDS. In this paper, we provide an in-depth analysis of existing metrics. Specifically, we analyze a typical cost-based scheme [6], and demonstrate that this approach is very confusing and ineffective when the cost factor is not carefully selected. In addition, we provide a novel information-theoretic analysis of IDS and propose a new metric that highly complements cost-based analysis. When examining the intrusion detection process from an information-theoretic point of view, intuitively, we should have less uncertainty about the input (event data) given the IDS output (alarm data). Thus, our new metric, CI D (Intrusion Detection Capability), is defined as the ratio of the mutual information between the IDS input and output to the entropy of the input. CI D has the desired property that: (1) It takes into account all the important aspects of detection capability naturally, i.e., true positive rate, false positive rate, positive predictive value, negative predictive value, and base rate; (2) it objectively provides an intrinsic measure of intrusion detection capability; and (3) it is sensitive to IDS operation parameters such as true positive rate and false positive rate, which can demonstrate the effect of the subtle changes of intrusion detection systems. We propose CI D as an appropriate performance measure to maximize when fine-tuning an IDS. The obtained operation point is the best that can be achieved by the IDS in terms of its intrinsic ability to classify input data. We use numerical examples as well as experiments of actual IDSs on various data sets to show that by using CI D, we can choose the best (optimal) operating point for an IDS and objectively compare different IDSs.