On the memory complexity of the forward-backward algorithm

Authors:
Wael Khreich;Eric Granger;Ali Miri;Robert Sabourin
Affiliations:
Laboratoire d'imagerie, de vision et d'intelligence artificielle (LIVIA), ícole de technologie supérieure, Montreal, QC, Canada;Laboratoire d'imagerie, de vision et d'intelligence artificielle (LIVIA), ícole de technologie supérieure, Montreal, QC, Canada;School of Information Technology and Engineering (SITE), University of Ottawa, Ottawa, ON, Canada;Laboratoire d'imagerie, de vision et d'intelligence artificielle (LIVIA), ícole de technologie supérieure, Montreal, QC, Canada
Venue:
Pattern Recognition Letters
Year:
2010

Citing 9
Cited 6

Matrix multiplication via arithmetic progressions

Journal of Symbolic Computation - Special issue on computational algebraic complexity
Machine learning techniques for the computer security domain of anomaly detection

Machine learning techniques for the computer security domain of anomaly detection
Incremental learning of discrete hidden markov models

Incremental learning of discrete hidden markov models
Exponential stability of filters and smoothers for hidden Markovmodels

IEEE Transactions on Signal Processing
Hidden Markov processes

IEEE Transactions on Information Theory
On receiver structures for channels having memory

IEEE Transactions on Information Theory
Compact encoding of stationary Markov sources

IEEE Transactions on Information Theory
Decision making in Markov chains applied to the problem of pattern recognition

IEEE Transactions on Information Theory
Baum's forward-backward algorithm revisited

Pattern Recognition Letters

Dynamic selection of generative-discriminative ensembles for off-line signature verification

Pattern Recognition
A survey of techniques for incremental learning of HMM parameters

Information Sciences: an International Journal
Gradient computation in linear-chain conditional random fields using the entropy message passing algorithm

Pattern Recognition Letters
Real-Time Tactical and Strategic Sales Management for Intelligent Agents Guided by Economic Regimes

Information Systems Research
Increasing mapping based hidden Markov model for dynamic process monitoring and diagnosis

Expert Systems with Applications: An International Journal
Comment on the paper "On the memory complexity of the forward-backward algorithm," by Khreich W., Granger E., Miri A., Sabourin, R.

Pattern Recognition Letters

Quantified Score

Hi-index	0.10

Visualization

Abstract

The Forward-backward (FB) algorithm forms the basis for estimation of Hidden Markov Model (HMM) parameters using the Baum-Welch technique. It is however, known to be prohibitively costly when estimation is performed from long observation sequences. Several alternatives have been proposed in literature to reduce the memory complexity of FB at the expense of increased time complexity. In this paper, a novel variation of the FB algorithm - called the Efficient Forward Filtering Backward Smoothing (EFFBS) - is proposed to reduce the memory complexity without the computational overhead. Given an HMM with N states and an observation sequence of length T, both FB and EFFBS algorithms have the same time complexity, O(N^2T). Nevertheless, FB has a memory complexity of O(NT), while EFFBS has a memory complexity that is independent of T, O(N). EFFBS requires fewer resources than FB, yet provides the same results.