Enhanced Sports Image Annotation and Retrieval Based Upon Semantic Analysis of Multimodal Cues

  • Authors:

  • Kraisak Kesorn;Stefan Poslad

  • Affiliations:

  • School of Electronic Engineering and Computer Science, Queen Mary University of London, London, United Kingdom E1 4NS;School of Electronic Engineering and Computer Science, Queen Mary University of London, London, United Kingdom E1 4NS

  • Venue:
  • PSIVT '09 Proceedings of the 3rd Pacific Rim Symposium on Advances in Image and Video Technology
  • Year:
  • 2009

Quantified Score

Hi-index 0.01

Visualization

Abstract

This paper presents a framework for semi-automatic annotation and semantic image retrieval, applied to the sports domain, based upon semantic analysis of both image text captions and visual features of the image. Unstructured text captions of images are analysed in order to extract the concepts and restructure them into a semantic model. SVM classification of the multi-dominant colours and edge ratio information of the images are used to classify the sport genre. The novelty of the proposed semantic framework is that it can find both the indirectly relevant concepts (concepts not directly referred to) in the visual information and can represent the semantic of images at a higher level by combining image captions and visual feature information. In addition, integrating LSI into the semantic framework enables the proposed system to tolerate ontology imperfections. Experimental results show that the use of the semantic approach significantly enhances image retrieval. Semantic visual information classification and retrieval based upon multimodal cues.