Optimal classifiers with minimum expected error within a Bayesian framework-Part I: Discrete and Gaussian models

Authors:
Lori A. Dalton;Edward R. Dougherty
Affiliations:
The Ohio State University, Department of Electrical and Computer Engineering, 205 Dreese Laboratory, 2015 Neil Avenue, Columbus, OH 43210, United States;Department of Electrical and Computer Engineering, Texas A&M University, College Station, TX, USA and Computational Biology Division, Translational Genomics Research Institute, Phoenix, AZ, USA
Venue:
Pattern Recognition
Year:
2013

Citing 14
Cited 2

Pattern Classification (2nd Edition)

Pattern Classification (2nd Edition)
Is cross-validation valid for small-sample microarray classification?

Bioinformatics
Optimal number of features as a function of sample size for various classification rules

Bioinformatics
Decorrelation of the true and estimated classifier errors in high-dimensional settings

EURASIP Journal on Bioinformatics and Systems Biology
The peaking phenomenon in the presence of feature-selection

Pattern Recognition Letters
On the sampling distribution of resubstitution and leave-one-out error estimators for linear classifiers

Pattern Recognition
Determination of the optimal number of features for quadratic discriminant analysis via the normal approximation to the discriminant distribution

Pattern Recognition
Application of the Bayesian MMSE estimator for classification error to gene expression microarray data

Bioinformatics
Optimal mean-square-error calibration of classifier error estimators under Bayesian models

Pattern Recognition
Bayesian Minimum Mean-Square Error Estimation for Classification Error—Part I: Definition and the Bayesian MMSE Error Estimator for Discrete Classification

IEEE Transactions on Signal Processing
Bayesian Minimum Mean-Square Error Estimation for Classification Error—Part II: Linear Classification of Gaussian Models

IEEE Transactions on Signal Processing
On the mean accuracy of statistical pattern recognizers

IEEE Transactions on Information Theory
Exact representation of the second-order moments for resubstitution and leave-one-out error estimation for linear discriminant analysis in the univariate heteroskedastic Gaussian model

Pattern Recognition
Analytic Study of Performance of Error Estimators for Linear Discriminant Analysis

IEEE Transactions on Signal Processing

Quantified Score

Hi-index	0.01

Visualization

Abstract

In recent years, biomedicine has faced a flood of difficult small-sample phenotype discrimination problems. A host of classification rules have been proposed to discriminate types of pathology, stages of disease and other diagnoses. Typically, these classification rules are heuristic algorithms, with very little understood about their performance. To give a concrete mathematical structure to the problem, recent work has utilized a Bayesian modeling framework based on an uncertainty class of feature-label distributions to both optimize and analyze error estimator performance. The current study uses the same Bayesian framework to also optimize classifier design. This completes a Bayesian theory of classification, where both the classifier error and the estimate of the error may be optimized and studied probabilistically within the model framework. This paper, the first of a two-part study, derives optimal classifiers in discrete and Gaussian models, demonstrates their superior performance over popular classifiers within the assumed model, and applies the method to real genomic data. The second part of the study discusses properties of these optimal Bayesian classifiers.