Connectionist learning of belief networks
Artificial Intelligence
Smooth on-line learning algorithms for hidden Markov models
Neural Computation
Backpropagation: the basic theory
Backpropagation
An HMM/MLP architecture for sequence recognition
Neural Computation
Neural Computation
Connectionist Speech Recognition: A Hybrid Approach
Connectionist Speech Recognition: A Hybrid Approach
THE APPLICATION OF STOCHASTIC CONTEXT-FREE GRAMMARS TO FOLDING, ALIGNING AND MODELING HOMOLOGOUS RNA SEQUENCES
Adaptive mixtures of local experts
Neural Computation
Neural Computation
Bidirectional Dynamics for Protein Secondary Structure Prediction
Sequence Learning - Paradigms, Algorithms, and Applications
The Journal of Machine Learning Research
Neural Networks - Special issue on neural networks and kernel methods for structured domains
2005 Speical Issue: Graph kernels for chemical informatics
Neural Networks - Special issue on neural networks and kernel methods for structured domains
Unsupervised learning of Bulgarian POS tags
MorphSlav '03 Proceedings of the 2003 EACL Workshop on Morphological Processing of Slavic Languages
Self-Organizing Hidden Markov Model Map (SOHMMM)
Neural Networks
| Hi-index | 0.00 |
We describe a hybrid modeling approach where the parameters of a model are calculated and modulated by another model, typically a neural network (NN), to avoid both overfitting and underfitting. We develop the approach for the case of Hidden Markov Models (HMMs), by deriving a class of hybrid HMM/NN architectures. These architectures can be trained with unified algorithms that blend HMM dynamic programming with NN backpropagation. In the case of complex data, mixtures of HMMs or modulated HMMs must be used. NNs can then be applied both to the parameters of each single HMM, and to the switching or modulation of the models, as a function of input or context. Hybrid HMM/NN architectures provide a flexible NN parameterization for the control of model structure and complexity. At the same time, they can capture distributions that, in practice, are inaccessible to single HMMs. The HMM/NN hybrid approach is tested, in its simplest form, by constructing a model of the immunoglobulin protein family. A hybrid model is trained, and a multiple alignment derived, with less than a fourth of the number of parameters used with previous single HMMs.