Efficient modeling of discrete events for anomaly detection using hidden markov models

  • Authors:
  • German Florez-Larrahondo;Susan M. Bridges;Rayford Vaughn

  • Affiliations:
  • Department of Computer Science and Engineering, Mississippi State University, Mississippi State, MS;Department of Computer Science and Engineering, Mississippi State University, Mississippi State, MS;Department of Computer Science and Engineering, Mississippi State University, Mississippi State, MS

  • Venue:
  • ISC'05 Proceedings of the 8th international conference on Information Security
  • Year:
  • 2005

Quantified Score

Hi-index 0.00

Visualization

Abstract

Anomaly detection systems are developed by learning a baseline-model from a set of events captured from a computer system operating under normal conditions. The model is then used to recognize unusual activities as deviations from normality. Hidden Markov models (HMMs) are powerful probabilistic finite state machines that have been used to acquire these baseline-models. Although previous research has indicated that HMMs can effectively represent complex sequences, the traditional learning algorithm for HMMs is too computationally expensive for use with real-world anomaly detection systems. This paper describes the use of a novel incremental learning algorithm for HMMs that allows the efficient acquisition of anomaly detection models. The new learning algorithm requires less memory and training time than previous approaches for learning discrete HMMs and can be used to perform online learning of accurate baseline-models from complex computer applications to support anomaly detection.