An optimization criterion for generalized discriminant analysis on undersampled problems

Authors:
Jieping Ye;R. Janardan;C. H. Park;H. Park
Affiliations:
Dept. of Comput. Sci. & Eng., Univ. of Minnesota-Twin Cities, Minneapolis, MN, USA;-;-;-
Venue:
IEEE Transactions on Pattern Analysis and Machine Intelligence
Year:
2004

Citing 0
Cited 63

IDR/QR: an incremental dimension reduction algorithm via QR decomposition

Proceedings of the tenth ACM SIGKDD international conference on Knowledge discovery and data mining
Using Uncorrelated Discriminant Analysis for Tissue Classification with Gene Expression Data

IEEE/ACM Transactions on Computational Biology and Bioinformatics (TCBB)
A Two-Stage Linear Discriminant Analysis via QR-Decomposition

IEEE Transactions on Pattern Analysis and Machine Intelligence
IDR/QR: An Incremental Dimension Reduction Algorithm via QR Decomposition

IEEE Transactions on Knowledge and Data Engineering
Capitalize on Dimensionality Increasing Techniques for Improving Face Recognition Grand Challenge Performance

IEEE Transactions on Pattern Analysis and Machine Intelligence
Null space versus orthogonal linear discriminant analysis

ICML '06 Proceedings of the 23rd international conference on Machine learning
Discriminant feature extraction using dual-objective optimization model

Pattern Recognition Letters
Data complexity assessment in undersampled classification of high-dimensional biomedical data

Pattern Recognition Letters
Feature Reduction via Generalized Uncorrelated Linear Discriminant Analysis

IEEE Transactions on Knowledge and Data Engineering
Graph Embedding and Extensions: A General Framework for Dimensionality Reduction

IEEE Transactions on Pattern Analysis and Machine Intelligence
The theoretical analysis of GLRAM and its applications

Pattern Recognition
Class-dependent PCA, MDC and LDA: A combined classifier for pattern classification

Pattern Recognition
Face recognition using a kernel fractional-step discriminant analysis algorithm

Pattern Recognition
Improved discriminate analysis for high-dimensional data and its application to face recognition

Pattern Recognition
Two-stage optimal component analysis

Computer Vision and Image Understanding
1D-LDA vs. 2D-LDA: When is vector-based linear discriminant analysis better than matrix-based?

Pattern Recognition
An evaluation of identity representability of facial expressions using feature distributions

Neurocomputing
Solution for supervised graph embedding: A case study

Signal Processing
Feature extraction using constrained maximum variance mapping

Pattern Recognition
Locally linear discriminant embedding: An efficient method for face recognition

Pattern Recognition
On Applying Dimension Reduction for Multi-labeled Problems

MLDM '07 Proceedings of the 5th international conference on Machine Learning and Data Mining in Pattern Recognition
General Solution for Supervised Graph Embedding

ECML '07 Proceedings of the 18th European conference on Machine Learning
A note on two-dimensional linear discriminant analysis

Pattern Recognition Letters
A Hybrid Nonlinear-Discriminant Analysis Feature Projection Technique

AI '08 Proceedings of the 21st Australasian Joint Conference on Artificial Intelligence: Advances in Artificial Intelligence
Fast algorithm for updating the discriminant vectors of dual-space LDA

IEEE Transactions on Information Forensics and Security
Ubiquitously supervised subspace learning

IEEE Transactions on Image Processing
A regularization framework for robust dimensionality reduction with applications to image reconstruction and feature extraction

Pattern Recognition
A new and fast implementation for null space based linear discriminant analysis

Pattern Recognition
Generalized discriminant analysis: a matrix exponential approach

IEEE Transactions on Systems, Man, and Cybernetics, Part B: Cybernetics
Letters: Robust kernel discriminant analysis and its application to feature extraction and recognition

Neurocomputing
Metric learning with feature decomposition for image categorization

Neurocomputing
Dimension reduction by a novel unified scheme using divergence analysis and genetic search

Digital Signal Processing
Diagnosis of liver diseases from P31 MRS data based on feature selection using genetic algorithm

LSMS/ICSEE'10 Proceedings of the 2010 international conference on Life system modeling and simulation and intelligent computing, and 2010 international conference on Intelligent computing for sustainable energy and environment: Part III
Linear projection methods in face recognition under unconstrained illuminations: a comparative study

CVPR'04 Proceedings of the 2004 IEEE computer society conference on Computer vision and pattern recognition
Uncorrelated trace ratio linear discriminant analysis for undersampled problems

Pattern Recognition Letters
From classifiers to discriminators: A nearest neighbor rule induced discriminant analysis

Pattern Recognition
Fast Algorithms for the Generalized Foley-Sammon Discriminant Analysis

SIAM Journal on Matrix Analysis and Applications
A New and Fast Orthogonal Linear Discriminant Analysis on Undersampled Problems

SIAM Journal on Scientific Computing
Locally Minimizing Embedding and Globally Maximizing Variance: Unsupervised Linear Difference Projection for Dimensionality Reduction

Neural Processing Letters
Orthogonal Complete Discriminant Locality Preserving Projections for Face Recognition

Neural Processing Letters
An optimization criterion for generalized marginal Fisher analysis on undersampled problems

International Journal of Automation and Computing
Blockwise projection matrix versus blockwise data on undersampled problems: Analysis, comparison and applications

Pattern Recognition
Face recognition based on the multi-scale local image structures

Pattern Recognition
Three-dimensional modular discriminant analysis (3DMDA): A new feature extraction approach for face recognition

Computers and Electrical Engineering
Weighted generalized kernel discriminant analysis using fuzzy memberships

WSEAS Transactions on Mathematics
Kernel uncorrelated discriminant analysis for radar target recognition

ICONIP'06 Proceedings of the 13th international conference on Neural Information Processing - Volume Part II
Face recognition by inverse fisher discriminant features

ICB'06 Proceedings of the 2006 international conference on Advances in Biometrics
Recent advances in subspace analysis for face recognition

SINOBIOMETRICS'04 Proceedings of the 5th Chinese conference on Advances in Biometric Person Authentication
The solution space for fisher discriminant analysis and the uniqueness under constraints

SINOBIOMETRICS'04 Proceedings of the 5th Chinese conference on Advances in Biometric Person Authentication
Fast calculation for fisher criteria in small sample size problem

SINOBIOMETRICS'04 Proceedings of the 5th Chinese conference on Advances in Biometric Person Authentication
Regularized orthogonal linear discriminant analysis

Pattern Recognition
Incremental complete LDA for face recognition

Pattern Recognition
Trace quotient problems revisited

ECCV'06 Proceedings of the 9th European conference on Computer Vision - Volume Part II
Incremental learning of complete linear discriminant analysis for face recognition

Knowledge-Based Systems
Novel Fisher discriminant classifiers

Pattern Recognition
On optimizing the sum of the Rayleigh quotient and the generalized Rayleigh quotient on the unit sphere

Computational Optimization and Applications
A rank-one update method for least squares linear discriminant analysis with concept drift

Pattern Recognition
Equivalence Between LDA/QR and Direct LDA

International Journal of Cognitive Informatics and Natural Intelligence
Modular discriminant analysis and its applications

Artificial Intelligence Review
Application of kernel-genetic algorithm as nonlinear feature selection in tropical wood species recognition system

Computers and Electronics in Agriculture
Structure preserving non-negative matrix factorization for dimensionality reduction

Computer Vision and Image Understanding
Complexity-reduced implementations of complete and null-space-based linear discriminant analysis

Neural Networks
Development and evaluation of a biomedical search engine using a predicate-based vector space model

Journal of Biomedical Informatics

Quantified Score

Hi-index	0.14

Visualization

Abstract

An optimization criterion is presented for discriminant analysis. The criterion extends the optimization criteria of the classical Linear Discriminant Analysis (LDA) through the use of the pseudoinverse when the scatter matrices are singular. It is applicable regardless of the relative sizes of the data dimension and sample size, overcoming a limitation of classical LDA. The optimization problem can be solved analytically by applying the Generalized Singular Value Decomposition (GSVD) technique. The pseudoinverse has been suggested and used for undersampled problems in the past, where the data dimension exceeds the number of data points. The criterion proposed in this paper provides a theoretical justification for this procedure. An approximation algorithm for the GSVD-based approach is also presented. It reduces the computational complexity by finding subclusters of each cluster and uses their centroids to capture the structure of each cluster. This reduced problem yields much smaller matrices to which the GSVD can be applied efficiently. Experiments on text data, with up to 7,000 dimensions, show that the approximation algorithm produces results that are close to those produced by the exact algorithm.