Tag completion based on belief theory and neighbor voting

• Authors: Amel Znaidia;Hervé Le Borgne;Céline Hudelot
• Venue: Proceedings of the 3rd ACM conference on International conference on multimedia retrieval
• Year: 2013

We address the problem of tag completion for automatic image annotation. Our method consists in two main steps: creating a list of "candidate tags" from the visual neighbors of the untagged image then using them as pieces of evidence to be combined to provide the final list of predicted tags. Both steps introduce a scheme to tackle with imprecision and uncertainty. First, a bag-of-words (BOW) signature is generated for each neighbor using local soft coding. Second, a sum-pooling operation across the BOW of the k nearest neighbors provides the list of "candidate tags". Finally, we use neighbors as pieces of evidence to be combined according to the Dempster's rule to predict the more relevant tags. The method is evaluated in the context of image classification and that of tag suggestion. The database used for visual neighbors search contains 1.2 million images extracted from Flickr. Classification is evaluated on the well known Pascal VOC 2007 and MIR Flickr datasets, on which we obtain similar or better results than the state-of-the-art. For tag suggestion, we manually annotated 241 queries. As well, we obtain competitive results on this task.