Stochastic differential equations (3rd ed.): an introduction with applications
Stochastic differential equations (3rd ed.): an introduction with applications
Neural Computation
| Hi-index | 0.00 |
We present an algorithm inspired by diffusion networks for learning the input/output mapping of temporal sequences with recurrent neural networks. Noise is added to the activation dynamics of the neurons of the hidden layer and annealed during learning of an output path probability distribution. Noise therefore plays the role of a learning parameter. We compare some results obtained on 2 temporal tasks with this "dynamic noise annealing" algorithm with other learning algorithms. Finally we discuss why adding noise to the state space variables can be better than adding stochasticity in the weight space.