Weakly supervised codebook learning by iterative label propagation with graph quantization

  • Authors:

  • Liujuan Cao;Rongrong Ji;Wei Liu;Hongxun Yao;Qi Tian

  • Affiliations:

  • Harbin Engineering University, Harbin 150001, China;Columbia University, New York City 10027, United States and Harbin Institute of Technology, Harbin 150001, China;Columbia University, New York City 10027, United States;Harbin Institute of Technology, Harbin 150001, China;University of Texas at San Antonio, San Antonio 78249-1644, United States

  • Venue:
  • Signal Processing
  • Year:
  • 2013

Quantified Score

Hi-index 0.08

Visualization

Abstract

Visual codebook serves as a fundamental component in many state-of-the-art visual search and object recognition systems. While most existing codebooks are built based solely on unsupervised patch quantization, there are few works exploited image labels to supervise its construction. The key challenge lies in the following: image labels are global, but patch supervision should be local. Such imbalanced supervision is beyond the scope of most existing supervised codebooks [9,10,12-15,29]. In this paper, we propose a weakly supervised codebook learning framework, which integrates image labels to supervise codebook building with two steps: the Label Propagation step propagates image labels into local patches by multiple instance learning and instance selection [20,21]. The Graph Quantization step integrates patch labels to build codebook using Mean Shift. Both steps are co-optimized in an Expectation Maximization framework: the E-phase selects the best patches that minimize the semantic distortions in quantization to propagate image labels; while the M-phase groups similar patches with related labels (modeled by WordNet [18]), which minimizes the visual distortions in quantization. In quantitative experiments, our codebook outperforms state-of-the-art unsupervised and supervised codebooks [1,10,11,25,29] using benchmark datasets.