The Trace Ratio Optimization Problem for Dimensionality Reduction

  • Authors:
  • T. T. Ngo;M. Bellalij;Y. Saad

  • Affiliations:
  • thango@cs.umn.edu and saad@cs.umn.edu;Mohammed.Bellalij@univ-valenciennes.fr;-

  • Venue:
  • SIAM Journal on Matrix Analysis and Applications
  • Year:
  • 2010

Quantified Score

Hi-index 0.00

Visualization

Abstract

This paper considers the problem of optimizing the ratio $\mathrm{Tr}[V^{T}AV]/\mathrm{Tr}[V^{T}BV]$ over all unitary matrices $V$ with $p$ columns, where $A,B$ are two positive definite matrices. This problem is common in supervised learning techniques. However, because its numerical solution is typically expensive it is often replaced by the simpler optimization problem which consists of optimizing $\mathrm{Tr}[V^{T}AV]$ under the constraint that $V^{T}BV=I$, the identity matrix. The goal of this paper is to examine this trace ratio optimization problem in detail, to consider different algorithms for solving it, and to illustrate the use of these algorithms for dimensionality reduction.