SRDA: An Efficient Algorithm for Large-Scale Discriminant Analysis

  • Authors:
  • Deng Cai;Xiaofei He;Jiawei Han

  • Affiliations:
  • -;-;-

  • Venue:
  • IEEE Transactions on Knowledge and Data Engineering
  • Year:
  • 2008

Quantified Score

Hi-index 0.00

Visualization

Abstract

Linear Discriminant Analysis (LDA) has been a popular method for extracting features which preserve class separability. It has been widely used in many fields of information processing. However, the computation of LDA involves dense matrices eigen-decomposition which can be computationally expensive both in time and memory. Specifically, LDA has $O(mnt+t^3)$ time complexity and requires $O(mn+mt+nt)$ memory, where $m$ is the number of samples, $n$ is the number of features and $t=\min(m,n)$. When both $m$ and $n$ are large, it is infeasible to apply LDA. In this paper, we propose a novel algorithm for discriminant analysis, called {\em Spectral Regression Discriminant Analysis} (SRDA). By using spectral graph analysis, SRDA casts discriminant analysis into a regression framework which facilitates both efficient computation and the use of regularization techniques. Specifically, SRDA only needs to solve a set of regularized least squares problems and there is no eigenvector computation involved, which is a huge save of both time and memory. Our theoretical analysis shows that SRDA can be computed with $O(ms)$ time and $O(ms)$ memory, where $s (\leq n)$ is the average number of non-zero features in each sample. Extensive experimental results on four real world data sets demonstrate the effectiveness and efficiency of our algorithm.