On one method of non-diagonal regularization in sparse Bayesian learning

Authors:
Dmitry Kropotov;Dmitry Vetrov
Affiliations:
Dorodnicyn Computing Centre of the Russian Academy of Sciences, Russia, Moscow;Moscow State University, Russia, Moscow
Venue:
Proceedings of the 24th international conference on Machine learning
Year:
2007

Citing 9
Cited 0

Bayesian regularization and pruning using a Laplace prior

Neural Computation
Approximate statistical tests for comparing supervised classification learning algorithms

Neural Computation
Bayesian Learning for Neural Networks

Bayesian Learning for Neural Networks
Bayesian parameter estimation via variational methods

Statistics and Computing
Variational Relevance Vector Machines

UAI '00 Proceedings of the 16th Conference on Uncertainty in Artificial Intelligence
Expectation Propagation for approximate Bayesian inference

UAI '01 Proceedings of the 17th Conference in Uncertainty in Artificial Intelligence
Sparse bayesian learning and the relevance vector machine

The Journal of Machine Learning Research
Pattern Recognition and Machine Learning (Information Science and Statistics)

Pattern Recognition and Machine Learning (Information Science and Statistics)
Gene selection in cancer classification using sparse logistic regression with Bayesian regularization

Bioinformatics

Quantified Score

Hi-index	0.00

Visualization

Abstract

In the paper we propose a new type of regularization procedure for training sparse Bayesian methods for classification. Transforming Hessian matrix of log-likelihood function to diagonal form with further regularization of its eigenvectors allows us to optimize evidence explicitly as a product of one-dimensional integrals. The process of automatic regularization coefficients determination then converges in one iteration. We show how to use the proposed approach for Gaussian and Laplace priors. Both algorithms show comparable performance with the state-of-the-art Relevance Vector Machines (RVM) but require less time for training and produce more sparse decision rules (in terms of degrees of freedom).