Online rare events detection

Authors:
Jun Hua Zhao;Xue Li;Zhao Yang Dong
Affiliations:
School of ITEE, The Univ of Queensland, St. Lucia, QLD, Australia;School of ITEE, The Univ of Queensland, St. Lucia, QLD, Australia;School of ITEE, The Univ of Queensland, St. Lucia, QLD, Australia
Venue:
PAKDD'07 Proceedings of the 11th Pacific-Asia conference on Advances in knowledge discovery and data mining
Year:
2007

Citing 7
Cited 2

MetaCost: a general method for making classifiers cost-sensitive

KDD '99 Proceedings of the fifth ACM SIGKDD international conference on Knowledge discovery and data mining
Mining time-changing data streams

Proceedings of the seventh ACM SIGKDD international conference on Knowledge discovery and data mining
Specification-based anomaly detection: a new approach for detecting network intrusions

Proceedings of the 9th ACM conference on Computer and communications security
Learning Quickly When Irrelevant Attributes Abound: A New Linear-Threshold Algorithm

Machine Learning
Mining concept-drifting data streams using ensemble classifiers

Proceedings of the ninth ACM SIGKDD international conference on Knowledge discovery and data mining
Mining with rarity: a unifying framework

ACM SIGKDD Explorations Newsletter - Special issue on learning from imbalanced datasets
The foundations of cost-sensitive learning

IJCAI'01 Proceedings of the 17th international joint conference on Artificial intelligence - Volume 2

Meta-learning for imbalanced data and classification ensemble in binary classification

Neurocomputing
A dynamic over-sampling procedure based on sensitivity for multi-class problems

Pattern Recognition

Quantified Score

Hi-index	0.00

Visualization

Abstract

Rare events detection is regarded as an imbalanced classification problem, which attempts to detect the events with high impact but low probability. Rare events detection has many applications such as network intrusion detection and credit fraud detection. In this paper we propose a novel online algorithm for rare events detection. Different from traditional accuracy-oriented approaches, our approach employs a number of hypothesis tests to perform the cost/benefit analysis. Our approach can handle online data with unbounded data volume by setting up a proper moving-window size and a forgetting factor. A comprehensive theoretical proof of our algorithm is given. We also conduct the experiments that achieve significant improvements compared with the most relevant algorithms based on publicly available real-world datasets.