Kernel-based nonlinear discriminant analysis using minimum squared errors criterion for multiclass and undersampled problems

Authors:
Wen-Jun Zeng;Xi-Lin Li;Xian-Da Zhang;En Cheng
Affiliations:
Key Laboratory of Underwater Acoustic Communication and Marine Information Technology of the Ministry of Education, Xiamen University, Xiamen 361005, China;Department of Computer Science and Electrical Engineering, University of Maryland Baltimore County, Baltimore, MD 21250, USA;Department of Automation, Tsinghua University, Beijing 100084, China;Key Laboratory of Underwater Acoustic Communication and Marine Information Technology of the Ministry of Education, Xiamen University, Xiamen 361005, China
Venue:
Signal Processing
Year:
2010

Citing 19
Cited 2

Introduction to statistical pattern recognition (2nd ed.)

Introduction to statistical pattern recognition (2nd ed.)
Eigenfaces vs. Fisherfaces: Recognition Using Class Specific Linear Projection

IEEE Transactions on Pattern Analysis and Machine Intelligence
Nonlinear component analysis as a kernel eigenvalue problem

Neural Computation
Nonlinear Fisher discriminant analysis using a minimum squared error cost function and the orthogonal least squares algorithm

Neural Networks
Structure Preserving Dimension Reduction for Clustered Text Data Based on the Generalized Singular Value Decomposition

SIAM Journal on Matrix Analysis and Applications
Pattern Classification (2nd Edition)

Pattern Classification (2nd Edition)
KPCA Plus LDA: A Complete Kernel Fisher Discriminant Framework for Feature Extraction and Recognition

IEEE Transactions on Pattern Analysis and Machine Intelligence
Nonlinear Discriminant Analysis Using Kernel Functions and the Generalized Singular Value Decomposition

SIAM Journal on Matrix Analysis and Applications
Characterization of a Family of Algorithms for Generalized Discriminant Analysis on Undersampled Problems

The Journal of Machine Learning Research
A Relationship between Linear Discriminant Analysis and the Generalized Minimum Squared~Error~Solution

SIAM Journal on Matrix Analysis and Applications
Generalized Discriminant Analysis Using a Kernel Approach

Neural Computation
Adaptive Nonlinear Discriminant Analysis by Regularized Minimum Squared Errors

IEEE Transactions on Knowledge and Data Engineering
Kernel Uncorrelated and Orthogonal Discriminant Analysis: A Unified Approach

CVPR '06 Proceedings of the 2006 IEEE Computer Society Conference on Computer Vision and Pattern Recognition - Volume 1
Least squares linear discriminant analysis

Proceedings of the 24th international conference on Machine learning
Nonlinear kernel-based statistical pattern analysis

IEEE Transactions on Neural Networks
An introduction to kernel-based learning algorithms

IEEE Transactions on Neural Networks
Face recognition using kernel direct discriminant analysis algorithms

IEEE Transactions on Neural Networks
Efficient and robust feature extraction by maximum margin criterion

IEEE Transactions on Neural Networks
Face recognition using kernel scatter-difference-based discriminant analysis

IEEE Transactions on Neural Networks

Multiclass classification machines with the complexity of a single binary classifier

Signal Processing
Multiple kernel local Fisher discriminant analysis for face recognition

Signal Processing

Quantified Score

Hi-index	0.08

Visualization

Abstract

It is well known that there exist two fundamental limitations in the linear discriminant analysis (LDA). One is that it cannot be applied when the within-class scatter matrix is singular, which is caused by the undersampled problem. The other is that it lacks the capability to capture the nonlinearly clustered structure of the data due to its linear nature. In this paper, a new kernel-based nonlinear discriminant analysis algorithm using minimum squared errors criterion (KDA-MSE) is proposed to solve these two problems. After mapping the original data into a higher-dimensional feature space using kernel function, the MSE criterion is used as the discriminant rule and the corresponding dimension reducing transformation is derived. Since the MSE solution does not require the scatter matrices to be nonsingular, the proposed KDA-MSE algorithm is applicable to the undersampled problem. Moreover, the new KDA-MSE algorithm can be applied to multiclass problem, whereas the existing MSE-based kernel discriminant methods are limited to handle twoclass data only. Extensive experiments, including object recognition and face recognition on three benchmark databases, are performed and the results demonstrate that our algorithm is competitive in comparison with other kernel-based discriminant techniques in terms of recognition accuracy.