Anomaly Detection over Noisy Data using Learned Probability Distributions
ICML '00 Proceedings of the Seventeenth International Conference on Machine Learning
Algorithmic Learning in a Random World
Algorithmic Learning in a Random World
Hedging Predictions in Machine Learning
The Computer Journal
A Tutorial on Conformal Prediction
The Journal of Machine Learning Research
ACM Computing Surveys (CSUR)
Novel data stream pattern mining report on the StreamKDD'10 workshop
ACM SIGKDD Explorations Newsletter
A study of anomaly detection in data from urban sensor networks
MDAI'12 Proceedings of the 9th international conference on Modeling Decisions for Artificial Intelligence
Knowledge discovery using genetic algorithm for maritime situational awareness
Expert Systems with Applications: An International Journal
Hi-index | 0.00 |
This paper presents a novel application of the theory of conformal prediction for distribution-independent on-line learning and anomaly detection. We exploit the fact that conformal predictors give valid prediction sets at specified confidence levels under the relatively weak assumption that the (normal) training data together with (normal) observations to be predicted have been generated from the same distribution. If the actual observation is not included in the possibly empty prediction set, it is classified as anomalous at the corresponding significance level. Interpreting the significance level as an upper bound of the probability that a normal observation is mistakenly classified as anomalous, we can conveniently adjust the sensitivity to anomalies while controlling the rate of false alarms without having to find any application specific thresholds. The proposed method has been evaluated in the domain of sea surveillance using recorded data assumed to be normal. The validity of the prediction sets is justified by the empirical error rate which is just below the significance level. In addition, experiments with simulated anomalous data indicate that anomaly detection sensitivity is superior to that of two previously proposed methods.