A Flexible and Efficient Algorithm for Regularized Fisher Discriminant Analysis

Authors:
Zhihua Zhang;Guang Dai;Michael I. Jordan
Affiliations:
College of Computer Science and Technology, Zhejiang University, Hangzhou, Zhejiang, China 310027;College of Computer Science and Technology, Zhejiang University, Hangzhou, Zhejiang, China 310027;Department of Statistics and Division of Computer Science, University of California, Berkeley, Berkeley, USA 94720
Venue:
ECML PKDD '09 Proceedings of the European Conference on Machine Learning and Knowledge Discovery in Databases: Part II
Year:
2009

Citing 11
Cited 5

Matrix analysis

Matrix analysis
Optimal Fisher discriminant analysis using the rank decomposition

Pattern Recognition
Structure Preserving Dimension Reduction for Clustered Text Data Based on the Generalized Singular Value Decomposition

SIAM Journal on Matrix Analysis and Applications
Pattern Classification (2nd Edition)

Pattern Classification (2nd Edition)
Kernel Methods for Pattern Analysis

Kernel Methods for Pattern Analysis
IDR/QR: an incremental dimension reduction algorithm via QR decomposition

Proceedings of the tenth ACM SIGKDD international conference on Knowledge discovery and data mining
Nonlinear Discriminant Analysis Using Kernel Functions and the Generalized Singular Value Decomposition

SIAM Journal on Matrix Analysis and Applications
A Relationship between Linear Discriminant Analysis and the Generalized Minimum Squared~Error~Solution

SIAM Journal on Matrix Analysis and Applications
Generalized Discriminant Analysis Using a Kernel Approach

Neural Computation
Least squares linear discriminant analysis

Proceedings of the 24th international conference on Machine learning
Generalizing discriminant analysis using the generalized singular value decomposition

IEEE Transactions on Pattern Analysis and Machine Intelligence

Robust kernel discriminant analysis using fuzzy memberships

Pattern Recognition
Model sparsity and brain pattern interpretation of classification models in neuroimaging

Pattern Recognition
Probabilistic Fisher discriminant analysis: A robust and flexible alternative to Fisher discriminant analysis

Neurocomputing
Model-based clustering of high-dimensional data: A review

Computational Statistics & Data Analysis
A Rayleigh-Ritz style method for large-scale discriminant analysis

Pattern Recognition

Quantified Score

Hi-index	0.00

Visualization

Abstract

Fisher linear discriminant analysis (LDA) and its kernel extension--kernel discriminant analysis (KDA)--are well known methods that consider dimensionality reduction and classification jointly. While widely deployed in practical problems, there are still unresolved issues surrounding their efficient implementation and their relationship with least mean squared error procedures. In this paper we address these issues within the framework of regularized estimation. Our approach leads to a flexible and efficient implementation of LDA as well as KDA. We also uncover a general relationship between regularized discriminant analysis and ridge regression. This relationship yields variations on conventional LDA based on the pseudoinverse and a direct equivalence to an ordinary least squares estimator. Experimental results on a collection of benchmark data sets demonstrate the effectiveness of our approach.