Flexible manifold embedding: a framework for semi-supervised and unsupervised dimension reduction

  • Authors:
  • Feiping Nie;Dong Xu;Ivor Wai-Hung Tsang;Changshui Zhang

  • Affiliations:
  • School of Computer Engineering, Nanyang Technological Univ., Singapore and State Key Laboratory of Intelligent Techn. and Systems Tsinghua National Laboratory for Inf. Science and Techn., Departme ...;School of Computer Engineering, Nanyang Technological University, Singapore;School of Computer Engineering, Nanyang Technological University, Singapore;State Key Laboratory of Intelligent Technologies and Systems Tsinghua National Laboratory for Information Science and Technology, Department of Automation, Tsinghua University, Beijing, China

  • Venue:
  • IEEE Transactions on Image Processing
  • Year:
  • 2010

Quantified Score

Hi-index 0.01

Visualization

Abstract

We propose a unified manifold learning framework for semi-supervised and unsupervised dimension reduction by employing a simple but effective linear regression function to map the new data points. For semi-supervised dimension reduction, we aim to find the optimal prediction labels Ffor all the training samples X, the linear regression function h(X) and the regression residue F0 = F - h(X) simultaneously. Our new objective function integrates two terms related to label fitness and manifold smoothness as well as a flexible penalty term defined on the residue F0. Our Semi-Supervised learning framework, referred to as flexible manifold embedding (FME), can effectively utilize label information from labeled data as well as a manifold structure from both labeled and unlabeled data. By modeling the mismatch between h(X) and F, we show that FME relaxes the hard linear constraint F = h(X) in manifold regularization (MR), making it better cope with the data sampled from a nonlinear manifold. In addition, we propose a simplified version (referred to as FME/U) for unsupervised dimension reduction.We also show that our proposed framework provides a unified view to explain and understand many semi-supervised, supervised and unsupervised dimension reduction techniques. Comprehensive experiments on several benchmark databases demonstrate the significant improvement over existing dimension reduction algorithms.