Nonlinear Bayesian Filters for Training Recurrent Neural Networks

Authors:
Ienkaran Arasaratnam;Simon Haykin
Affiliations:
Cognitive Systems Laboratory Department of Electrical & Computer Engineering, McMaster University, Hamilton, L8S 4K1;Cognitive Systems Laboratory Department of Electrical & Computer Engineering, McMaster University, Hamilton, L8S 4K1
Venue:
MICAI '08 Proceedings of the 7th Mexican International Conference on Artificial Intelligence: Advances in Artificial Intelligence
Year:
2008

Citing 11
Cited 2

Comparative Analysis of Backpropagation and the Extended Kalman Filter for Training Multilayer Perceptrons

IEEE Transactions on Pattern Analysis and Machine Intelligence
Original Contribution: Optimal filtering algorithms for fast learning in feedforward neural networks

Neural Networks
Bayesian forecasting and dynamic models (2nd ed.)

Bayesian forecasting and dynamic models (2nd ed.)
Kalman Filtering and Neural Networks

Kalman Filtering and Neural Networks
Neural Networks: A Comprehensive Foundation

Neural Networks: A Comprehensive Foundation
Neural Network Control of Nonlinear Discrete-Time Systems (Public Administration and Public Policy)

Neural Network Control of Nonlinear Discrete-Time Systems (Public Administration and Public Policy)
Monte Carlo Strategies in Scientific Computing

Monte Carlo Strategies in Scientific Computing
Nonlinear adaptive prediction of nonstationary signals

IEEE Transactions on Signal Processing
New developments in state estimation for nonlinear systems

Automatica (Journal of IFAC)
On the Kalman filtering method in neural network training and pruning

IEEE Transactions on Neural Networks
Training Recurrent Neurocontrollers for Robustness With Derivative-Free Kalman Filter

IEEE Transactions on Neural Networks

Prediction for noisy nonlinear time series by echo state network based on dual estimation

Neurocomputing
An algorithm for on-line detection of high frequency oscillations related to epilepsy

Computer Methods and Programs in Biomedicine

Quantified Score

Hi-index	0.00

Visualization

Abstract

In this paper, we present nonlinear Bayesian filters for training recurrent neural networks with a special emphasis on a novel, more accurate, derivative-free member of the approximate Bayesian filter family called the cubature Kalman filter. We discuss the theory of Bayesian filters, which is rooted in the state-space modeling of the dynamic system in question and the linear estimation principle. For improved numerical stability and optimal performance during training period, a number of techniques of how to tune Bayesian filters is suggested. We compare the predictability of various Bayesian filter-trained recurrent neural networks using a chaotic time-series. From the empirical results, we conclude that the performance may be greatly improved by the new square-root cubature Kalman filter.