Matrix computations (3rd ed.)
Database-friendly random projections
PODS '01 Proceedings of the twentieth ACM SIGMOD-SIGACT-SIGART symposium on Principles of database systems
Event Detection and Analysis from Video Streams
IEEE Transactions on Pattern Analysis and Machine Intelligence
Random projection in dimensionality reduction: applications to image and text data
Proceedings of the seventh ACM SIGKDD international conference on Knowledge discovery and data mining
Coupled hidden Markov models for complex action recognition
CVPR '97 Proceedings of the 1997 Conference on Computer Vision and Pattern Recognition (CVPR '97)
Clustering Large Graphs via the Singular Value Decomposition
Machine Learning
Diagnosing network-wide traffic anomalies
Proceedings of the 2004 conference on Applications, technologies, architectures, and protocols for computer communications
Topic transition detection using hierarchical hidden Markov and semi-Markov models
Proceedings of the 13th annual ACM international conference on Multimedia
IEEE Transactions on Knowledge and Data Engineering
Example-Based Robust Outlier Detection in High Dimensional Datasets
ICDM '05 Proceedings of the Fifth IEEE International Conference on Data Mining
Effective and Efficient Dimensionality Reduction for Large-Scale and Streaming Data Preprocessing
IEEE Transactions on Knowledge and Data Engineering
Decentralized compression and predistribution via randomized gossiping
Proceedings of the 5th international conference on Information processing in sensor networks
Fast Monte Carlo Algorithms for Matrices II: Computing a Low-Rank Approximation to a Matrix
SIAM Journal on Computing
Hierarchical Anomaly Detection in Distributed Large-Scale Sensor Networks
ISCC '06 Proceedings of the 11th IEEE Symposium on Computers and Communications
Robust Real-Time Unusual Event Detection using Multiple Fixed-Location Monitors
IEEE Transactions on Pattern Analysis and Machine Intelligence
Unsupervised Learning of Human Action Categories Using Spatial-Temporal Words
International Journal of Computer Vision
Anomaly Detection Support Vector Machine and Its Application to Fault Diagnosis
ICDM '08 Proceedings of the 2008 Eighth IEEE International Conference on Data Mining
ACM Computing Surveys (CSUR)
Compressive-Projection Principal Component Analysis and the First Eigenvector
DCC '09 Proceedings of the 2009 Data Compression Conference
An iterative image registration technique with an application to stereo vision
IJCAI'81 Proceedings of the 7th international joint conference on Artificial intelligence - Volume 2
Effective Anomaly Detection in Sensor Networks Data Streams
ICDM '09 Proceedings of the 2009 Ninth IEEE International Conference on Data Mining
A fast outlier detection strategy for distributed high-dimensional data sets with mixed attributes
Data Mining and Knowledge Discovery
Data Mining and Knowledge Discovery
TACO: tunable approximate computation of outliers in wireless sensor networks
Proceedings of the 2010 ACM SIGMOD International Conference on Management of data
Anomaly detection in IP networks
IEEE Transactions on Signal Processing
Optimized Projections for Compressed Sensing
IEEE Transactions on Signal Processing
FRaC: a feature-modeling approach for semi-supervised and unsupervised anomaly detection
Data Mining and Knowledge Discovery
IEEE Transactions on Information Theory
IEEE Transactions on Information Theory
Near-Optimal Signal Recovery From Random Projections: Universal Encoding Strategies?
IEEE Transactions on Information Theory
| Hi-index | 0.00 |
This paper addresses the anomaly detection problem in large-scale data mining applications using residual subspace analysis. We are specifically concerned with situations where the full data cannot be practically obtained due to physical limitations such as low bandwidth, limited memory, storage, or computing power. Motivated by the recent compressed sensing (CS) theory, we suggest a framework wherein random projection can be used to obtained compressed data, addressing the scalability challenge. Our theoretical contribution shows that the spectral property of the CS data is approximately preserved under a such a projection and thus the performance of spectral-based methods for anomaly detection is almost equivalent to the case in which the raw data is completely available. Our second contribution is the construction of the framework to use this result and detect anomalies in the compressed data directly, thus circumventing the problems of data acquisition in large sensor networks. We have conducted extensive experiments to detect anomalies in network and surveillance applications on large datasets, including the benchmark PETS 2007 and 83 GB of real footage from three public train stations. Our results show that our proposed method is scalable, and importantly, its performance is comparable to conventional methods for anomaly detection when the complete data is available.