Annotation of paintings with high-level semantic concepts using transductive inference and ontology-based concept disambiguation

  • Authors:
  • Liza Leslie;Tat-Seng Chua;Jain Ramesh

  • Affiliations:
  • National University of Singapore, Singapore, Singapore;National University of Singapore, Singapore, Singapore;UC Irvine, Irvine, CA

  • Venue:
  • Proceedings of the 15th international conference on Multimedia
  • Year:
  • 2007

Quantified Score

Hi-index 0.00

Visualization

Abstract

Domain-specific knowledge of paintings defines a wide range of concepts for annotation and flexible retrieval of paintings. In this work, we employ the ontology of artistic concepts that includes visual (or atomic) concepts at the intermediate level and high-level concepts at the application level. Visual-level color and brushwork concepts are widely used by art historians to analyze paintings and serve as cues for annotating high-level concepts such as the artist names, painting styles and art periods for paintings. In this research we combine the color and brushwork concepts with low-level features and utilize the transductive inference framework to annotate high-level concepts to the image blocks. In order to resolve conflicting assignments of high-level concepts, we further employ the ontology-based concept disambiguation method and generate image-level annotations. This method performs global optimization of the block-level annotations using the linear constraints extracted from domain knowledge. Our experiments on annotating high-level concepts demonstrate that: a) the use of visual-level concepts significantly improves the accuracy as compared to using low-level features only; and b) the proposed transductive inference framework out-performs the conventional baseline methods and c) the proposed ontology-based disambiguation method generates superior results for several annotation scenarios.