Distinguishing the unexplainable from the merely unusual: adding explanations to outliers to discover and detect significant complex rare events

Authors:
Ted E. Senator;Henry G. Goldberg;Alex Memory
Affiliations:
SAIC;SAIC;SAIC
Venue:
Proceedings of the ACM SIGKDD Workshop on Outlier Detection and Description
Year:
2013

Citing 5
Cited 0

Ongoing management and application of discovered knowledge in a large regulatory organization: a case study of the use and impact of NASD Regulation's Advanced Detection System (RADS)

Proceedings of the sixth ACM SIGKDD international conference on Knowledge discovery and data mining
The NASD regulation advanced detection system: integrating data mining and visualization for break detection in the NASDAQ stock market

Information visualization in data mining and knowledge discovery
Multi-Stage Classification

ICDM '05 Proceedings of the Fifth IEEE International Conference on Data Mining
Apolo: making sense of large network data by combining rich user interaction and machine learning

Proceedings of the SIGCHI Conference on Human Factors in Computing Systems
Use of Domain Knowledge to Detect Insider Threats in Computer Activities

SPW '13 Proceedings of the 2013 IEEE Security and Privacy Workshops

Quantified Score

Hi-index	0.00

Visualization

Abstract

This paper discusses the key role of explanations for applications that discover and detect significant complex rare events. These events are distinguished not necessarily by outliers (i.e., unusual or rare data values), but rather by their inexplicability in terms of appropriate real-world behaviors. Outlier detection techniques are typically part of such applications and may provide useful starting points; however, they are far from sufficient for identifying events of interest and discriminating them from similar but uninteresting events to a degree necessary for operational utility. Other techniques that distinguish anomalies from outliers, and then enable anomalies to be classified as relevant or not to the particular detection problem are also necessary. We argue that explanations are the key to the effectiveness of such complex rare event detection applications, and illustrate this point with examples from several real applications.