On Consistency of Bayesian Inference with Mixtures of Logistic Regression

Authors:
Yang Ge;Wenxin Jiang
Affiliations:
Department of Statistics, Northwestern University, Evanston, IL 60208, U.S.A.;Department of Statistics, Northwestern University, Evanston, IL 60208, U.S.A.
Venue:
Neural Computation
Year:
2006

Citing 4
Cited 3

Hierarchical mixtures of experts and the EM algorithm

Neural Computation
Convergence results for the EM approach to mixtures of experts architectures

Neural Networks
On the approximation rate of hierarchical mixtures-of-experts for generalized linear models

Neural Computation
Consistency of posterior distributions for neural networks

Neural Networks

A note on mixtures of experts for multiclass responses: approximation rate and Consistent Bayesian Inference

ICML '06 Proceedings of the 23rd international conference on Machine learning
On the Consistency of Bayesian Variable Selection for High Dimensional Binary Regression and Classification

Neural Computation
On convergence rates of mixtures of polynomial experts

Neural Computation

Quantified Score

Hi-index	0.00

Visualization

Abstract

This is a theoretical study of the consistency properties of Bayesian inference using mixtures of logistic regression models. When standard logistic regression models are combined in a mixtures-of-experts setup, a flexible model is formed to model the relationship between a binary (yes-no) response y and a vector of predictors x. Bayesian inference conditional on the observed data can then be used for regression and classification. This letter gives conditions on choosing the number of experts (i.e., number of mixing components) k or choosing a prior distribution for k, so that Bayesian inference is consistent, in the sense of often approximating the underlying true relationship between y and x. The resulting classification rule is also consistent, in the sense of having near-optimal performance in classification. We show these desirable consistency properties with a nonstochastic k growing slowly with the sample size n of the observed data, or with a random k that takes large values with nonzero but small probabilities.