A new perspective to null linear discriminant analysis method and its fast implementation using random matrix multiplication with scatter matrices

Authors:
Alok Sharma;Kuldip K. Paliwal
Affiliations:
Signal Processing Lab, Griffith University, Brisbane, Australia and Human Genome Center, Institute of Medical Science, University of Tokyo, Tokyo, Japan and School of Engineering & Physics, Univer ...;Signal Processing Lab, Griffith University, Brisbane, Australia
Venue:
Pattern Recognition
Year:
2012

Citing 19
Cited 4

Matrix multiplication via arithmetic progressions

Journal of Symbolic Computation - Special issue on computational algebraic complexity
Introduction to statistical pattern recognition (2nd ed.)

Introduction to statistical pattern recognition (2nd ed.)
Using Discriminant Eigenfeatures for Image Retrieval

IEEE Transactions on Pattern Analysis and Machine Intelligence
Eigenfaces vs. Fisherfaces: Recognition Using Class Specific Linear Projection

IEEE Transactions on Pattern Analysis and Machine Intelligence
Expected classification error of the Fisher linear classifier with pseudo-inverse covariance matrix

Pattern Recognition Letters
Fractional-Step Dimensionality Reduction

IEEE Transactions on Pattern Analysis and Machine Intelligence
Recognizing Imprecisely Localized, Partially Occluded, and Expression Variant Faces from a Single Sample per Class

IEEE Transactions on Pattern Analysis and Machine Intelligence
Solving the Small Sample Size Problem of LDA

ICPR '02 Proceedings of the 16 th International Conference on Pattern Recognition (ICPR'02) Volume 3 - Volume 3
Discriminative Common Vectors for Face Recognition

IEEE Transactions on Pattern Analysis and Machine Intelligence
Characterization of a Family of Algorithms for Generalized Discriminant Analysis on Undersampled Problems

The Journal of Machine Learning Research
Computational and Theoretical Analysis of Null Space and Orthogonal Linear Discriminant Analysis

The Journal of Machine Learning Research
A comparison of generalized linear discriminant analysis algorithms

Pattern Recognition
A Gradient Linear Discriminant Analysis for Small Sample Sized Problem

Neural Processing Letters
Eigenfeature Regularization and Extraction in Face Recognition

IEEE Transactions on Pattern Analysis and Machine Intelligence
Bayes Optimality in Linear Discriminant Analysis

IEEE Transactions on Pattern Analysis and Machine Intelligence
Cancer classification by gradient LDA technique using microarray gene expression data

Data & Knowledge Engineering
Asymmetric Principal Component and Discriminant Analyses for Pattern Classification

IEEE Transactions on Pattern Analysis and Machine Intelligence
A new and fast implementation for null space based linear discriminant analysis

Pattern Recognition
Generalizing discriminant analysis using the generalized singular value decomposition

IEEE Transactions on Pattern Analysis and Machine Intelligence

A two-stage linear discriminant analysis for face-recognition

Pattern Recognition Letters
Complexity-reduced implementations of complete and null-space-based linear discriminant analysis

Neural Networks
A feature selection method using fixed-point algorithm for DNA microarray gene expression data

International Journal of Knowledge-based and Intelligent Engineering Systems
A feature selection method using improved regularized linear discriminant analysis

Machine Vision and Applications

Quantified Score

Hi-index	0.01

Visualization

Abstract

Null linear discriminant analysis (LDA) method is a popular dimensionality reduction method for solving small sample size problem. The implementation of null LDA method is, however, computationally very expensive. In this paper, we theoretically derive the null LDA method from a different perspective and present a computationally efficient implementation of this method. Eigenvalue decomposition (EVD) of S"T^+S"B (where S"B is the between-class scatter matrix and S"T^+ is the pseudoinverse of the total scatter matrix S"T) is shown here to be a sufficient condition for the null LDA method. As EVD of S"T^+S"Bis computationally expensive, we show that the utilization of random matrix together with S"T^+S"B is also a sufficient condition for null LDA method. This condition is used here to derive a computationally fast implementation of the null LDA method. We show that the computational complexity of the proposed implementation is significantly lower than the other implementations of the null LDA method reported in the literature. This result is also confirmed by conducting classification experiments on several datasets.