Multi-class Transductive Learning Based on ℓ1 Relaxations of Cheeger Cut and Mumford-Shah-Potts Model

  • Authors:
  • Xavier Bresson;Xue-Cheng Tai;Tony F. Chan;Arthur Szlam

  • Affiliations:
  • Department of Computer Science, City University of Hong Kong, Kowloon Tang, Hong Kong;Department of Mathematics, University of Bergen, Bergen, Norway;Department of Mathematics and Computer Science, Hong Kong University of Science and Technology, Kowloon, Hong Kong;Department of Mathematics, The City College of New York, New York, USA

  • Venue:
  • Journal of Mathematical Imaging and Vision
  • Year:
  • 2014

Quantified Score

Hi-index 0.00

Visualization

Abstract

Recent advances in ℓ1 optimization for imaging problems provide promising tools to solve the fundamental high-dimensional data classification in machine learning. In this paper, we extend the main result of Szlam and Bresson (Proceedings of the 27th International Conference on Machine Learning, pp. 1039---1046, 2010), which introduced an exact ℓ1 relaxation of the Cheeger ratio cut problem for unsupervised data classification. The proposed extension deals with the multi-class transductive learning problem, which consists in learning several classes with a set of labels for each class. Learning several classes (i.e. more than two classes) simultaneously is generally a challenging problem, but the proposed method builds on strong results introduced in imaging to overcome the multi-class issue. Besides, the proposed multi-class transductive learning algorithms also benefit from recent fast ℓ1 solvers, specifically designed for the total variation norm, which plays a central role in our approach. Finally, experiments demonstrate that the proposed ℓ1 relaxation algorithms are more accurate and robust than standard ℓ2 relaxation methods s.a. spectral clustering, particularly when considering a very small number of labels for each class to be classified. For instance, the mean error of classification for the benchmark MNIST dataset of 60,000 data in $\mathbb{R}^{784}$ using the proposed ℓ1 relaxation of the multi-class Cheeger cut is 2.4聽% when only one label is considered for each class, while the error of classification for the ℓ2 relaxation method of spectral clustering is 24.7聽%.