Quantifying tag representativeness of visual content of social images

• Authors: Aixin Sun;Sourav S. Bhowmick
• Venue: Proceedings of the international conference on Multimedia
• Year: 2010

Social tags describe images from many aspects including the visual content observable from the images, the context and usage of images, user opinions and others. Not all tags are therefore useful for image search and are appropriate for tag recommendation with respect to visual content of images. However, the relationship between a given tag and the visual content of its tagged images are largely ignored in existing studies on tags and in tagging applications. In this paper, we bridge the two orthogonal areas of social image tagging and query performance prediction in Web search, to quantify tag representativeness of the visual content presented in the annotated images, which is also known as tag visual-representativeness. In simple words, tag visual-representativeness characterizes the effectiveness of a tag in describing the visual content of the set of images annotated by the tag. A tag is visually representative if its annotated images are visually similar to each other, containing a common visual concept such as an object or a scene. We propose two distance metrics, namely cohesion and separation, to quantify tag visual-representativeness from the set of images annotated by a tag and the entire image collection. Through extensive experiments on a subset of Flickr images, we demonstrate the characteristics of seven variants of the distance metrics derived from different low-level image representations and show that the visually representative tags can be identified with high precision. Importantly, these proposed distance measures are parameter free with linear or constant computational complexity, thus are effective for practical applications.