System identification: theory for the user
System identification: theory for the user
Multilayer feedforward networks are universal approximators
Neural Networks
Introduction to the theory of neural computation
Introduction to the theory of neural computation
Stochastic differential equations (3rd ed.): an introduction with applications
Stochastic differential equations (3rd ed.): an introduction with applications
Fundamentals of speech recognition
Fundamentals of speech recognition
Toward a theory of information processing in graded, random, and interactive networks
Attention and performance XIV (silver jubilee volume)
An introduction to signal detection and estimation (2nd ed.)
An introduction to signal detection and estimation (2nd ed.)
A learning theorem for networks at detailed stochastic equilibrium
Neural Computation
Learning path distributions using nonequilibrium diffusion networks
NIPS '97 Proceedings of the 1997 conference on Advances in neural information processing systems 10
Proceedings of the 1998 conference on Advances in neural information processing systems II
Learning nonlinear dynamical systems using an EM algorithm
Proceedings of the 1998 conference on Advances in neural information processing systems II
Neural Networks for Pattern Recognition
Neural Networks for Pattern Recognition
A Tutorial on Support Vector Machines for Pattern Recognition
Data Mining and Knowledge Discovery
Learning Dynamical Models Using Expectation-Maximisation
ICCV '98 Proceedings of the Sixth International Conference on Computer Vision
Robust Contour Tracking in Echocardiographic Sequences
ICCV '98 Proceedings of the Sixth International Conference on Computer Vision
Gradient calculations for dynamic recurrent neural networks: a survey
IEEE Transactions on Neural Networks
A Continuous Restricted Boltzmann Machine with a Hardware-Amenable Learning Algorithm
ICANN '02 Proceedings of the International Conference on Artificial Neural Networks
Learning from biological neurons to compute with electronic noise
Proceedings of the International Conference on Computer-Aided Design
| Hi-index | 0.00 |
We present a Monte Carlo approach for training partially observable diffusion processes. We apply the approach to diffusion networks, a stochastic version of continuous recurrent neural networks. The approach is aimed at learning probability distributions of continuous paths, not just expected values. Interestingly, the relevant activation statistics used by the learning rule presented here are inner products in the Hilbert space of square integrable functions. These inner products can be computed using Hebbian operations and do not require backpropagation of error signals. Moreover, standard kernel methods could potentially be applied to compute such inner products. We propose that the main reason that recurrent neural networks have not worked well in engineering applications (e.g., speech recognition) is that they implicitly rely on a very simplistic likelihood model. The diffusion network approach proposed here is much richer and may open new avenues for applications of recurrent neural networks. We present some analysis and simulations to support this view. Very encouraging results were obtained on a visual speech recognition task in which neural networks outperformed hidden Markov models.