One-class conditional random fields for sequential anomaly detection

Authors:
Yale Song;Zhen Wen;Ching-Yung Lin;Randall Davis
Affiliations:
MIT Computer Science and Artificial Intelligence Laboratory;IBM T.J. Watson Research Center;IBM T.J. Watson Research Center;MIT Computer Science and Artificial Intelligence Laboratory
Venue:
IJCAI'13 Proceedings of the Twenty-Third international joint conference on Artificial Intelligence
Year:
2013

Citing 16
Cited 2

LOF: identifying density-based local outliers

SIGMOD '00 Proceedings of the 2000 ACM SIGMOD international conference on Management of data
Conditional Random Fields: Probabilistic Models for Segmenting and Labeling Sequence Data

ICML '01 Proceedings of the Eighteenth International Conference on Machine Learning
Latent dirichlet allocation

The Journal of Machine Learning Research
Support Vector Data Description

Machine Learning
Kernel conditional random fields: representation and clique selection

ICML '04 Proceedings of the twenty-first international conference on Machine learning
Estimating the Support of a High-Dimensional Distribution

Neural Computation
Outlier detection by active learning

Proceedings of the 12th ACM SIGKDD international conference on Knowledge discovery and data mining
Hidden Conditional Random Fields

IEEE Transactions on Pattern Analysis and Machine Intelligence
Anomaly detection: A survey

ACM Computing Surveys (CSUR)
Learning structural SVMs with latent variables

ICML '09 Proceedings of the 26th Annual International Conference on Machine Learning
Anomaly detection for symbolic sequences and time series data

Anomaly detection for symbolic sequences and time series data
Sequential anomaly detection based on temporal-difference learning: Principles, models and case studies

Applied Soft Computing
Bundle Methods for Regularized Risk Minimization

The Journal of Machine Learning Research
PLDA+: Parallel latent dirichlet allocation with data placement and pipeline processing

ACM Transactions on Intelligent Systems and Technology (TIST)
LIBSVM: A library for support vector machines

ACM Transactions on Intelligent Systems and Technology (TIST)
Regularized bundle methods for convex and non-convex risks

The Journal of Machine Learning Research

Automated image analysis framework for high-throughput determination of grapevine berry sizes using conditional random fields

Computers and Electronics in Agriculture
A least-squares approach to anomaly detection in static and sequential data

Pattern Recognition Letters

Quantified Score

Hi-index	0.00

Visualization

Abstract

Sequential anomaly detection is a challenging problem due to the one-class nature of the data (i.e., data is collected from only one class) and the temporal dependence in sequential data. We present One-Class Conditional Random Fields (OCCRF) for sequential anomaly detection that learn from a one-class dataset and capture the temporal dependence structure, in an unsupervised fashion. We propose a hinge loss in a regularized risk minimization framework that maximizes the margin between each sequence being classified as "normal" and "abnormal." This allows our model to accept most (but not all) of the training data as normal, yet keeps the solution space tight. Experimental results on a number of real-world datasets show our model outperforming several baselines. We also report an exploratory study on detecting abnormal organizational behavior in enterprise social networks.