On optimizing kernel-based fisher discriminant analysis using prototype reduction schemes

Authors:
Sang-Woon Kim;B. John Oommen
Affiliations:
Dept. of Computer Science and Engineering, Myongji University, Yongin, Korea;School of Computer Science, Carleton University, Ottawa, ON, Canada
Venue:
SSPR'06/SPR'06 Proceedings of the 2006 joint IAPR international conference on Structural, Syntactic, and Statistical Pattern Recognition
Year:
2006

Citing 11
Cited 0

Introduction to statistical pattern recognition (2nd ed.)

Introduction to statistical pattern recognition (2nd ed.)
Nonlinear component analysis as a kernel eigenvalue problem

Neural Computation
Fast computation of low rank matrix approximations

STOC '01 Proceedings of the thirty-third annual ACM symposium on Theory of computing
Two Variations on Fisher's Linear Discriminant for Pattern Recognition

IEEE Transactions on Pattern Analysis and Machine Intelligence
A Tutorial on Support Vector Machines for Pattern Recognition

Data Mining and Knowledge Discovery
Nonlinear Fisher discriminant analysis using a minimum squared error cost function and the orthogonal least squares algorithm

Neural Networks
Sparse Greedy Matrix Approximation for Machine Learning

ICML '00 Proceedings of the Seventeenth International Conference on Machine Learning
Linear Dimensionality Reduction via a Heteroscedastic Extension of LDA: The Chernoff Criterion

IEEE Transactions on Pattern Analysis and Machine Intelligence
KPCA Plus LDA: A Complete Kernel Fisher Discriminant Framework for Feature Extraction and Recognition

IEEE Transactions on Pattern Analysis and Machine Intelligence
Generalized Discriminant Analysis Using a Kernel Approach

Neural Computation
Finding Prototypes For Nearest Neighbor Classifiers

IEEE Transactions on Computers

Quantified Score

Hi-index	0.00

Visualization

Abstract

Fisher’s Linear Discriminant Analysis (LDA) is a traditional dimensionality reduction method that has been proven to be successful for decades. Numerous variants, such as the Kernel-based Fisher Discriminant Analysis (KFDA) have been proposed to enhance the LDA’s power for nonlinear discriminants. Though effective, the KFDA is computationally expensive, since the complexity increases with the size of the data set. In this paper, we suggest a novel strategy to enhance the computation for an entire family of KFDA’s. Rather than invoke the KFDA for the entire data set, we advocate that the data be first reduced into a smaller representative subset using a Prototype Reduction Scheme (PRS), and that dimensionality reduction be achieved by invoking a KFDA on this reduced data set. In this way data points which are ineffective in the dimension reduction and classification can be eliminated to obtain a significantly reduced kernel matrix, K, without degrading the performance. Our experimental results demonstrate that the proposed mechanism dramatically reduces the computation time without sacrificing the classification accuracy for artificial and real-life data sets.