Fast likelihood search for hidden Markov models

  • Authors:
  • Yasuhiro Fujiwara;Yasushi Sakurai;Masaru Kitsuregawa

  • Affiliations:
  • NTT Cyber Space Labs, Yokosuka-Shi, Kanagawa, Japan and University of Tokyo, Meguro-ku, Tokyo, Japan;NTT Communication Science Labs, Soraku, Kyoto, Japan;University of Tokyo, Meguro-ku, Tokyo, Japan

  • Venue:
  • ACM Transactions on Knowledge Discovery from Data (TKDD)
  • Year:
  • 2009

Quantified Score

Hi-index 0.00

Visualization

Abstract

Hidden Markov models (HMMs) are receiving considerable attention in various communities and many applications that use HMMs have emerged such as mental task classification, biological analysis, traffic monitoring, and anomaly detection. This article has two goals; The first goal is exact and efficient identification of the model whose state sequence has the highest likelihood for the given query sequence (more precisely, no HMM that actually has a high-probability path for the given sequence is missed by the algorithm), and the second goal is exact and efficient monitoring of streaming data sequences to find the best model. We propose SPIRAL, a fast search method for HMM datasets. SPIRAL is based on three ideas; (1) it clusters states of models to compute approximate likelihood, (2) it uses several granularities and approximates likelihood values in search processing, and (3) it focuses on just the promising likelihood computations by pruning out low-likelihood state sequences. Experiments verify the effectiveness of SPIRAL and show that it is more than 490 times faster than the naive method.