Matrix computations (3rd ed.)
Bayesian Learning for Neural Networks
Bayesian Learning for Neural Networks
Sparse bayesian learning and the relevance vector machine
The Journal of Machine Learning Research
Pattern Recognition and Machine Learning (Information Science and Statistics)
Pattern Recognition and Machine Learning (Information Science and Statistics)
Latent-Space Variational Bayes
IEEE Transactions on Pattern Analysis and Machine Intelligence
Nonlinear time series online prediction using reservoir Kalman filter
IJCNN'09 Proceedings of the 2009 international joint conference on Neural Networks
Inferring parameters and structure of latent variable models by variational bayes
UAI'99 Proceedings of the Fifteenth conference on Uncertainty in artificial intelligence
Variational relevance vector machines
UAI'00 Proceedings of the Sixteenth conference on Uncertainty in artificial intelligence
Space-alternating generalized expectation-maximization algorithm
IEEE Transactions on Signal Processing
Survey: Reservoir computing approaches to recurrent neural network training
Computer Science Review
Channel parameter estimation in mobile radio environments using the SAGE algorithm
IEEE Journal on Selected Areas in Communications
IEEE Transactions on Signal Processing
IEEE Transactions on Signal Processing
Hi-index | 0.00 |
In this work, a variational Bayesian framework for efficient training of echo state networks (ESNs) with automatic regularization and delay&sum (D&S) readout adaptation is proposed. The algorithm uses a classical batch learning of ESNs. By treating the network echo states as fixed basis functions parameterized with delay parameters, we propose a variational Bayesian ESN training scheme. The variational approach allows for a seamless combination of sparse Bayesian learning ideas and a variational Bayesian space-alternating generalized expectation-maximization (VB-SAGE) algorithm for estimating parameters of superimposed signals. While the former method realizes automatic regularization of ESNs, which also determines which echo states and input signals are relevant for "explaining" the desired signal, the latter method provides a basis for joint estimation of D&S readout parameters. The proposed training algorithm can naturally be extended to ESNs with fixed filter neurons. It also generalizes the recently proposed expectation-maximization-based D&S readout adaptation method. The proposed algorithm was tested on synthetic data prediction tasks as well as on dynamic handwritten character recognition.