On Dimensionality, Sample Size, Classification Error, and Complexity of Classification Algorithm in Pattern Recognition

Authors:
Sarunas Raudys;Vitalijus Pikelis
Affiliations:
Lietuvos RSR Moksly, Adademiha, Lenino, U.S.S.R.;Lietuvos RSR Moksly, Adademiha, Lenino, U.S.S.R.
Venue:
IEEE Transactions on Pattern Analysis and Machine Intelligence
Year:
1980

Citing 0
Cited 9

A tree-based decision rule for identifying profile groups of cases without predefined classes: application in diffuse large B-cell lymphomas

Computers in Biology and Medicine
Independent rule in classification of multivariate binary data

Journal of Multivariate Analysis
Determination of the optimal number of features for quadratic discriminant analysis via the normal approximation to the discriminant distribution

Pattern Recognition
Is feature selection still necessary?

SLSFS'05 Proceedings of the 2005 international conference on Subspace, Latent Structure and Feature Selection
Exact representation of the second-order moments for resubstitution and leave-one-out error estimation for linear discriminant analysis in the univariate heteroskedastic Gaussian model

Pattern Recognition
Collective selection of the best version of a pattern recognition system

Pattern Recognition Letters
Effects of data set features on the performances of classification algorithms

Expert Systems with Applications: An International Journal
Resampling methods for quality assessment of classifier performance and optimal number of features

Signal Processing
Learning algorithms may perform worse with increasing training set size: Algorithm-data incompatibility

Computational Statistics & Data Analysis

Quantified Score

Hi-index	0.15

Visualization

Abstract

This paper compares four classification algorithms-discriminant functions when classifying individuals into two multivariate populations. The discriminant functions (DF's) compared are derived according to the Bayes rule for normal populations and differ in assumptions on the covariance matrices' structure. Analytical formulas for the expected probability of misclassification EPN are derived and show that the classification error EPN depends on the structure of a classification algorithm, asymptotic probability of misclassification P驴, and the ratio of learning sample size N to dimensionality p:N/p for all linear DF's discussed and N2/p for quadratic DF's. The tables for learning quantity H = EPN/P驴 depending on parameters P驴, N, and p for four classifilcation algorithms analyzed are presented and may be used for estimating the necessary learning sample size, detennining the optimal number of features, and choosing the type of the classification algorithm in the case of a limited learning sample size.