Blockwise projection matrix versus blockwise data on undersampled problems: Analysis, comparison and applications

Authors:
Zhizheng Liang;Shixiong Xia;Yong Zhou;Youfu Li
Affiliations:
Department of Computer Science, China University of Mining and Technology, XuZhou, PR China;Department of Computer Science, China University of Mining and Technology, XuZhou, PR China;Department of Computer Science, China University of Mining and Technology, XuZhou, PR China;Department of Manufacturing Engineering and Engineering Management, City University of Hong Kong, Hong Kong, PR China
Venue:
Pattern Recognition
Year:
2011

Citing 28
Cited 3

Application of the Karhunen-Loeve Procedure for the Characterization of Human Faces

IEEE Transactions on Pattern Analysis and Machine Intelligence
Introduction to statistical pattern recognition (2nd ed.)

Introduction to statistical pattern recognition (2nd ed.)
Eigenfaces vs. Fisherfaces: Recognition Using Class Specific Linear Projection

IEEE Transactions on Pattern Analysis and Machine Intelligence
PCA versus LDA

IEEE Transactions on Pattern Analysis and Machine Intelligence
A Unified Approach to Parallel Block-Jacobi Methods for the Symmetric Eigenvalue Problem

VECPAR '98 Selected Papers and Invited Talks from the Third International Conference on Vector and Parallel Processing
Two-Dimensional PCA: A New Approach to Appearance-Based Face Representation and Recognition

IEEE Transactions on Pattern Analysis and Machine Intelligence
The CMU Pose, Illumination, and Expression Database

IEEE Transactions on Pattern Analysis and Machine Intelligence
Gabor-Based Kernel PCA with Fractional Power Polynomial Models for Face Recognition

IEEE Transactions on Pattern Analysis and Machine Intelligence
Discriminative Common Vectors for Face Recognition

IEEE Transactions on Pattern Analysis and Machine Intelligence
Face Recognition Using Laplacianfaces

IEEE Transactions on Pattern Analysis and Machine Intelligence
A Two-Stage Linear Discriminant Analysis via QR-Decomposition

IEEE Transactions on Pattern Analysis and Machine Intelligence
Graph Embedding: A General Framework for Dimensionality Reduction

CVPR '05 Proceedings of the 2005 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'05) - Volume 2 - Volume 02
Clustered Blockwise PCA for Representing Visual Data

IEEE Transactions on Pattern Analysis and Machine Intelligence
Characterization of a Family of Algorithms for Generalized Discriminant Analysis on Undersampled Problems

The Journal of Machine Learning Research
Generalized Low Rank Approximations of Matrices

Machine Learning
Combining Reconstructive and Discriminative Subspace Methods for Robust Classification and Regression by Subsampling

IEEE Transactions on Pattern Analysis and Machine Intelligence
Subclass Discriminant Analysis

IEEE Transactions on Pattern Analysis and Machine Intelligence
Graph Embedding and Extensions: A General Framework for Dimensionality Reduction

IEEE Transactions on Pattern Analysis and Machine Intelligence
The theoretical analysis of GLRAM and its applications

Pattern Recognition
Equivalence of Some Common Linear Feature Extraction Techniques for Appearance-Based Object Recognition Tasks

IEEE Transactions on Pattern Analysis and Machine Intelligence
Discriminant Subspace Analysis: A Fukunaga-Koontz Approach

IEEE Transactions on Pattern Analysis and Machine Intelligence
Eigenfaces for recognition

Journal of Cognitive Neuroscience
A note on two-dimensional linear discriminant analysis

Pattern Recognition Letters
Rapid and brief communication: Two-dimensional FLD for face recognition

Pattern Recognition
A face and palmprint recognition approach based on discriminant DCT feature extraction

IEEE Transactions on Systems, Man, and Cybernetics, Part B: Cybernetics
Robust linear dimensionality reduction

IEEE Transactions on Visualization and Computer Graphics
An optimization criterion for generalized discriminant analysis on undersampled problems

IEEE Transactions on Pattern Analysis and Machine Intelligence
Generalizing discriminant analysis using the generalized singular value decomposition

IEEE Transactions on Pattern Analysis and Machine Intelligence

Adaptive weighted learning for linear regression problems via Kullback-Leibler divergence

Pattern Recognition
Computational and space complexity analysis of SubXPCA

Pattern Recognition
Training Lp norm multiple kernel learning in the primal

Neural Networks

Quantified Score

Hi-index	0.01

Visualization

Abstract

Linear subspace methods are extensively used in many areas such as pattern recognition and machine learning. Among them, block subspace methods are efficient in terms of the computational complexity. In this paper, we perform a thorough analysis on block subspace methods and give a theoretical framework for understanding block subspace methods. It reveals the relationship between block subspace methods and classical subspace methods. We theoretically show that blockwise PCA has larger reconstruction errors than classical PCA and classical LDA has stronger discriminant power than blockwise LDA in the case of the same number of reduced features. In addition, based on the Fisher criterion, we also give a strategy for selecting an approximate block size for classification problems. The comprehensive experiments on face images and gene expression data are used to evaluate our results and a comparative analysis for various methods is made. Experimental results demonstrate that overly combining subspaces of block subspace methods without considering the subspace distance may yield undesirable performance on undersampled problems.