A tight bound on the performance of Fisher's linear discriminant in randomly projected data spaces

Authors:
Robert John Durrant;Ata Kabán
Affiliations:
School of Computer Science, University of Birmingham, Edgbaston, B15 2TT, UK;School of Computer Science, University of Birmingham, Edgbaston, B15 2TT, UK
Venue:
Pattern Recognition Letters
Year:
2012

Citing 8
Cited 0

Random projection in dimensionality reduction: applications to image and text data

Proceedings of the seventh ACM SIGKDD international conference on Knowledge discovery and data mining
An elementary proof of a theorem of Johnson and Lindenstrauss

Random Structures & Algorithms
Learning Mixtures of Gaussians

FOCS '99 Proceedings of the 40th Annual Symposium on Foundations of Computer Science
Database-friendly random projections: Johnson-Lindenstrauss with binary coins

Journal of Computer and System Sciences - Special issu on PODS 2001
Experiments with random projections for machine learning

Proceedings of the ninth ACM SIGKDD international conference on Knowledge discovery and data mining
An algorithmic theory of learning: Robust concepts and random projection

Machine Learning
Compressed fisher linear discriminant analysis: classification of randomly projected data

Proceedings of the 16th ACM SIGKDD international conference on Knowledge discovery and data mining
Experiments with random projection

UAI'00 Proceedings of the Sixteenth conference on Uncertainty in artificial intelligence

Quantified Score

Hi-index	0.10

Visualization

Abstract

We consider the problem of classification in non-adaptive dimensionality reduction. Specifically, we give an average-case bound on the classification error of Fisher's linear discriminant classifier when the classifier only has access to randomly projected versions of a given training set. By considering the system of random projection and classifier together as a whole, we are able to take advantage of the simple class structure inherent in the problem, and so derive a non-trivial performance bound without imposing any sparsity or underlying low-dimensional structure restrictions on the data. Our analysis also reveals and quantifies the effect of class 'flipping' - a potential issue when randomly projecting a finite sample. Our bound is reasonably tight, and unlike existing bounds on learning from randomly projected data, it becomes tighter as the quantity of training data increases. A preliminary version of this work received an IBM Best Student Paper Award at the 20th International Conference on Pattern Recognition.