On the construction of the relevance vector machine based on Bayesian Ying-Yang harmony learning

Authors:
Dansong Cheng;Minh Nhut Nguyen;Junbin Gao;Daming Shi
Affiliations:
-;-;-;-
Venue:
Neural Networks
Year:
2013

Citing 12
Cited 0

Bayesian Ying-Yang machine, clustering and number of clusters

Pattern Recognition Letters - special issue on pattern recognition in practice V
Unsupervised Learning of Finite Mixture Models

IEEE Transactions on Pattern Analysis and Machine Intelligence
BYY harmony learning, structural RPCL, and topological self-organizing on mixture models

Neural Networks - New developments in self-organizing maps
Sparse bayesian learning and the relevance vector machine

The Journal of Machine Learning Research
Sensitivity analysis applied to the construction of radial basis function networks

Neural Networks
Smooth relevance vector machine: a smoothness prior extension of the RVM

Machine Learning
Improving the generalization properties of radial basis function neural networks

Neural Computation
Local regularization assisted orthogonal least squares regression

Neurocomputing
A Least-squares Approach to Direct Importance Estimation

The Journal of Machine Learning Research
Fuzzy CMAC with incremental Bayesian Ying-Yang learning and dynamic rule construction

IEEE Transactions on Systems, Man, and Cybernetics, Part B: Cybernetics
FCMAC-BYY: Fuzzy CMAC Using Bayesian Ying–Yang Learning

IEEE Transactions on Systems, Man, and Cybernetics, Part B: Cybernetics
Advances on BYY harmony learning: information theoretic perspective, generalized projection geometry, and independent factor autodetermination

IEEE Transactions on Neural Networks

Quantified Score

Hi-index	0.00

Visualization

Abstract

Tipping's relevance vector machine (RVM) applies kernel methods to construct basis function networks using a least number of relevant basis functions. Compared to the well-known support vector machine (SVM), the RVM provides a better sparsity, and an automatic estimation of hyperparameters. However, the performance of the original RVM purely depends on the smoothness of the presumed prior of the connection weights and parameters. Consequently, the sparsity is actually still controlled by the choice of kernel functions and/or kernel parameters. This may lead to severe underfitting or overfitting in some cases. In this research, we explicitly involve the number of basis functions into the objective of the optimization procedure, and construct the RVM by maximizing the harmony function between ''hypothetical'' probability distribution in the forward training pathway and ''true'' probability distribution in the backward testing pathway, using Xu's Bayesian Ying-Yang (BYY) harmony learning technique. The experimental results have shown that our proposed methodology can achieve both the least complexity of structure and goodness of fit to data.