A Hybrid Probabilistic Model for Unified Collaborative and Content-Based Image Tagging

Authors:
Ning Zhou;William K. Cheung;Guoping Qiu;Xiangyang Xue
Affiliations:
University of North Carolina, Charlotte;Hong Kong Baptist University, Hong Kong;University of Nottingham, Nottingham;Fudan University, Shanghai
Venue:
IEEE Transactions on Pattern Analysis and Machine Intelligence
Year:
2011

Citing 0
Cited 7

Tag suggestion and localization for web videos by bipartite graph matching

WSM '11 Proceedings of the 3rd ACM SIGMM international workshop on Social media
Geo-based automatic image annotation

Proceedings of the 2nd ACM International Conference on Multimedia Retrieval
Grass, scrub, trees and random forest

Proceedings of the 1st ACM international workshop on Multimedia analysis for ecological data
An efficient two-stage framework for image annotation

Pattern Recognition
Random forest for image annotation

ECCV'12 Proceedings of the 12th European conference on Computer Vision - Volume Part VI
Integrating low-level and semantic features for object consistent segmentation

Neurocomputing
Support vector description of clusters for content-based image annotation

Pattern Recognition

Quantified Score

Hi-index	0.14

Visualization

Abstract

The increasing availability of large quantities of user contributed images with labels has provided opportunities to develop automatic tools to tag images to facilitate image search and retrieval. In this paper, we present a novel hybrid probabilistic model (HPM) which integrates low-level image features and high-level user provided tags to automatically tag images. For images without any tags, HPM predicts new tags based solely on the low-level image features. For images with user provided tags, HPM jointly exploits both the image features and the tags in a unified probabilistic framework to recommend additional tags to label the images. The HPM framework makes use of the tag-image association matrix (TIAM). However, since the number of images is usually very large and user-provided tags are diverse, TIAM is very sparse, thus making it difficult to reliably estimate tag-to-tag co-occurrence probabilities. We developed a collaborative filtering method based on nonnegative matrix factorization (NMF) for tackling this data sparsity issue. Also, an L_1 norm kernel method is used to estimate the correlations between image features and semantic concepts. The effectiveness of the proposed approach has been evaluated using three databases containing 5,000 images with 371 tags, 31,695 images with 5,587 tags, and 269,648 images with 5,018 tags, respectively.