An efficient algorithm for generalized discriminant analysis using incomplete Cholesky decomposition

Authors:
Haixian Wang;Zilan Hu;Yu'e Zhao
Affiliations:
Research Center for Learning Science, Southeast University, Sipailou 2, Nanjing, Jiangsu 210096, PR China;School of Mathematics and Physics, Anhui University of Technology, Maanshan, Anhui 243002, PR China;Department of Mathematics, Qingdao University, Qingdao, Shandong 266071, PR China
Venue:
Pattern Recognition Letters
Year:
2007

Citing 11
Cited 1

Nonlinear component analysis as a kernel eigenvalue problem

Neural Computation
Nonlinear Fisher discriminant analysis using a minimum squared error cost function and the orthogonal least squares algorithm

Neural Networks
Latent Semantic Kernels

Journal of Intelligent Information Systems
Sparse Greedy Matrix Approximation for Machine Learning

ICML '00 Proceedings of the Seventeenth International Conference on Machine Learning
Kernel Eigenfaces vs. Kernel Fisherfaces: Face Recognition Using Kernel Methods

FGR '02 Proceedings of the Fifth IEEE International Conference on Automatic Face and Gesture Recognition
Kernel independent component analysis

The Journal of Machine Learning Research
A modified algorithm for generalized discriminant analysis

Neural Computation
Generalized Discriminant Analysis Using a Kernel Approach

Neural Computation
Null space-based kernel fisher discriminant analysis for face recognition

FGR' 04 Proceedings of the Sixth IEEE international conference on Automatic face and gesture recognition
Improving kernel Fisher discriminant analysis for face recognition

IEEE Transactions on Circuits and Systems for Video Technology
Foley-Sammon optimal discriminant vectors using kernel approach

IEEE Transactions on Neural Networks

Unsupervised feature extraction via kernel subspace techniques

Neurocomputing

Quantified Score

Hi-index	0.10

Visualization

Abstract

Generalized discriminant analysis (GDA) has provided an extremely powerful approach to extracting nonlinear features via kernel trick. And it has been suggested for a number of applications, such as classification problem. Whereas the GDA could be solved by the utilization of Mercer kernels, a drawback of the standard GDA is that it may suffer from computational problem for large scale data set. Besides, there is still attendant problem of numerical accuracy when computing the eigenvalue problem of large matrices. Also, the GDA would occupy large memory (to store the kernel matrix). To overcome these deficiencies, we use Gram-Schmidt orthonormalization and incomplete Cholesky decomposition to find a basis for the entire training samples, and then formulate GDA as another eigenvalue problem of matrix whose size is much smaller than that of the kernel matrix by using the basis, while still working out the optimal discriminant vectors from all training samples. The theoretical analysis and experimental results on both artificial and real data set have shown the superiority of the proposed method for performing GDA in terms of computational efficiency and even the recognition accuracy, especially when the training samples size is large.