An incremental Bayesian approach for training multilayer perceptrons

Authors:
Dimitris Tzikas;Aristidis Likas
Affiliations:
Biomedical Research Institute, FORTH, University Campus of Ioannina, Ioannina, Greece and Department of Computer Science, University of Ioannina, Ioannina, Greece;Biomedical Research Institute, FORTH, University Campus of Ioannina, Ioannina, Greece and Department of Computer Science, University of Ioannina, Ioannina, Greece
Venue:
ICANN'10 Proceedings of the 20th international conference on Artificial neural networks: Part I
Year:
2010

Citing 4
Cited 0

Bayesian Learning for Neural Networks

Bayesian Learning for Neural Networks
Sparse bayesian learning and the relevance vector machine

The Journal of Machine Learning Research
Smooth relevance vector machine: a smoothness prior extension of the RVM

Machine Learning
Sparse Bayesian modeling with adaptive kernel learning

IEEE Transactions on Neural Networks

Quantified Score

Hi-index	0.00

Visualization

Abstract

The multilayer perceptron (MLP) is a well established neural network model for supervised learning problems. Furthermore, it is well known that its performance for a given problem depends crucially on appropriately selecting the MLP architecture, which is typically achieved using cross-validation. In this work, we propose an incremental Bayesian methodology to address the important problem of automatic determination of the number of hidden units in MLPs with one hidden layer. The proposed methodology treats the one-hidden layer MLP as a linear model consisting of a weighted combination of basis functions (hidden units). Then an incremental method for sparse Bayesian learning of linear models is employed that effectively adjusts not only the combination weights, but also the parameters of the hidden units. Experimental results for several well-known classification data sets demonstrate that the proposed methodology successfully identifies optimal MLP architectures in terms of generalization error.