Structural Bayesian Linear Regression for Hidden Markov Models

Authors:
Shinji Watanabe;Atsushi Nakamura;Biing-Hwang (Fred) Juang
Affiliations:
Mitsubishi Electric Research Laboratories (MERL), Cambridge, USA;NTT Communication Science Laboratories, NTT Corporation, Kyoto, Japan;Center for Signal and Image Processing, Georgia Institute of Technology, Atlanta, USA
Venue:
Journal of Signal Processing Systems
Year:
2014

Citing 12
Cited 0

A view of the EM algorithm that justifies incremental, sparse, and other variants

Learning in graphical models
An Introduction to Variational Methods for Graphical Models

Machine Learning
Bayesian model search for mixture models based on optimizing variational bounds

Neural Networks
Improved Bayesian Learning of Hidden Markov Models for Speaker Adaptation

ICASSP '97 Proceedings of the 1997 IEEE International Conference on Acoustics, Speech, and Signal Processing (ICASSP '97)-Volume 2 - Volume 2
Machine Learning: Discriminative and Generative (Kluwer International Series in Engineering and Computer Science)

Machine Learning: Discriminative and Generative (Kluwer International Series in Engineering and Computer Science)
Variational Message Passing

The Journal of Machine Learning Research
On transforming statistical models for non-frontal face verification

Pattern Recognition
Adaptation of pitch and spectrum for HMM-based speech synthesis using MLLR

ICASSP '01 Proceedings of the Acoustics, Speech, and Signal Processing, 200. on IEEE International Conference - Volume 02
Static and dynamic variance compensation for recognition of reverberant speech with dereverberation preprocessing

IEEE Transactions on Audio, Speech, and Language Processing
Unsupervised Activity Recognition with User's Physical Characteristics Data

ISWC '11 Proceedings of the 2011 15th Annual International Symposium on Wearable Computers
Inferring parameters and structure of latent variable models by variational bayes

UAI'99 Proceedings of the Fifteenth conference on Uncertainty in artificial intelligence
Discriminative Training for Large-Vocabulary Speech Recognition Using Minimum Classification Error

IEEE Transactions on Audio, Speech, and Language Processing

Quantified Score

Hi-index	0.00

Visualization

Abstract

Linear regression for Hidden Markov Model (HMM) parameters is widely used for the adaptive training of time series pattern analysis especially for speech processing. The regression parameters are usually shared among sets of Gaussians in HMMs where the Gaussian clusters are represented by a tree. This paper realizes a fully Bayesian treatment of linear regression for HMMs considering this regression tree structure by using variational techniques. This paper analytically derives the variational lower bound of the marginalized log-likelihood of the linear regression. By using the variational lower bound as an objective function, we can algorithmically optimize the tree structure and hyper-parameters of the linear regression rather than heuristically tweaking them as tuning parameters. Experiments on large vocabulary continuous speech recognition confirm the generalizability of the proposed approach, especially when the amount of adaptation data is limited.