Outlier detection over sliding windows for probabilistic data streams

  • Authors:
  • Bin Wang;Xiao-Chun Yang;Guo-Ren Wang;Ge Yu

  • Affiliations:
  • School of Information Science and Engineering, Northeastern University, Shenyang, China and Key Laboratory of Medical Image Computing, Northeastern University, Ministry of Education, Shenyang, Chi ...;School of Information Science and Engineering, Northeastern University, Shenyang, China and Key Laboratory of Medical Image Computing, Northeastern University, Ministry of Education, Shenyang, Chi ...;School of Information Science and Engineering, Northeastern University, Shenyang, China and Key Laboratory of Medical Image Computing, Northeastern University, Ministry of Education, Shenyang, Chi ...;School of Information Science and Engineering, Northeastern University, Shenyang, China and Key Laboratory of Medical Image Computing, Northeastern University, Ministry of Education, Shenyang, Chi ...

  • Venue:
  • Journal of Computer Science and Technology
  • Year:
  • 2010

Quantified Score

Hi-index 0.00

Visualization

Abstract

Outlier detection is a very useful technique in many applications, where data is generally uncertain and could be described using probability. While having been studied intensively in the field of deterministic data, outlier detection is still novel in the emerging uncertain data field. In this paper, we study the semantic of outlier detection on probabilistic data stream and present a new definition of distance-based outlier over sliding window. We then show the problem of detecting an outlier over a set of possible world instances is equivalent to the problem of finding the k-th element in its neighborhood. Based on this observation, a dynamic programming algorithm (DPA) is proposed to reduce the detection cost from O(2|R(e,d)|) to O(|kċR(e,d)|), where R(e, d) is the d-neighborhood of e. Furthermore, we propose a pruning-based approach (PBA) to effectively and efficiently filter non-outliers on single window, and dynamically detect recent m elements incrementally. Finally, detailed analysis and thorough experimental results demonstrate the efficiency and scalability of our approach.