Bayesian multiscale smoothing in supervised and semi-supervised kernel discriminant analysis

Authors:
Subhadeep Mukhopadhyay;Anil K. Ghosh
Affiliations:
Department of Statistics, Texas A & M University, 3143 TAMU, College Station, TX 77843-3143, USA;Theoretical Statistics and Mathematics Unit, Indian Statistical Institute, 203, Barrackpore Trunk Road, Kolkata 700108, India
Venue:
Computational Statistics & Data Analysis
Year:
2011

Citing 10
Cited 0

A Classification EM algorithm for clustering and two stochastic versions

Computational Statistics & Data Analysis - Special issue on optimization techniques in statistics
Efficient Approximations for the MarginalLikelihood of Bayesian Networks with Hidden Variables

Machine Learning - Special issue on learning with probabilistic representations
A fast algorithm for the minimum covariance determinant estimator

Technometrics
On Bias, Variance, 0/1—Loss, and the Curse-of-Dimensionality

Data Mining and Knowledge Discovery
Semisupervised Learning of Classifiers: Theory, Algorithms, and Their Application to Human-Computer Interaction

IEEE Transactions on Pattern Analysis and Machine Intelligence
On Visualization and Aggregation of Nearest Neighbor Classifiers

IEEE Transactions on Pattern Analysis and Machine Intelligence
A Bayesian approach to bandwidth selection for multivariate kernel density estimation

Computational Statistics & Data Analysis
A classification EM algorithm for binned data

Computational Statistics & Data Analysis
The relative value of labeled and unlabeled samples in pattern recognition with an unknown mixing parameter

IEEE Transactions on Information Theory - Part 2
Stochastic Relaxation, Gibbs Distributions, and the Bayesian Restoration of Images

IEEE Transactions on Pattern Analysis and Machine Intelligence

Quantified Score

Hi-index	0.03

Visualization

Abstract

In kernel discriminant analysis, it is common practice to select the smoothing parameter (bandwidth) based on the training data and use it for classifying all unlabeled observations. But this method of selecting a single scale of smoothing ignores the major issue of model uncertainty. Moreover, in addition to depending on the training sample, a good choice of bandwidth may also depend on the observation to be classified, and a fixed level of smoothing may not work well in all parts of the measurement space. So, instead of using a single smoothing parameter, it may be more useful in practice to study classification results for multiple scales of smoothing and judiciously aggregate them to arrive at the final decision. This paper adopts a Bayesian approach to carry out one such multiscale analysis using a probabilistic framework. This framework also helps us to extend our multiscale method for semi-supervised classification, where, in addition to the training sample, one uses unlabeled test set observations to form the decision rule. Some well-known benchmark data sets are analyzed to show the utility of these proposed methods.