Auto-annotation of paintings using social annotations,domain ontology and transductive inference

Authors:
Liza Marchenko Leslie;Tat-Seng Chua;Ramesh Jain
Affiliations:
National University of Singapore;National University of Singapore;University of California, Irvine
Venue:
PCM'07 Proceedings of the multimedia 8th Pacific Rim conference on Advances in multimedia information processing
Year:
2007

Citing 8
Cited 0

Ontology-Based Photo Annotation

IEEE Intelligent Systems
Retrieval of Paintings using Effects Induced by Color Features

CAIVD '98 Proceedings of the 1998 International Workshop on Content-Based Access of Image and Video Databases (CAIVD '98)
Using syntactic dependency as local context to resolve word sense ambiguity

ACL '98 Proceedings of the 35th Annual Meeting of the Association for Computational Linguistics and Eighth Conference of the European Chapter of the Association for Computational Linguistics
Estimation of Dependences Based on Empirical Data: Springer Series in Statistics (Springer Series in Statistics)

Estimation of Dependences Based on Empirical Data: Springer Series in Statistics (Springer Series in Statistics)
Semantic role labeling via integer linear programming inference

COLING '04 Proceedings of the 20th international conference on Computational Linguistics
Exploiting full parsing information to label semantic roles using an ensemble of ME and SVM via integer linear programming

CONLL '05 Proceedings of the Ninth Conference on Computational Natural Language Learning
Ontology-based annotation of paintings using transductive inference framework

MMM'07 Proceedings of the 13th international conference on Multimedia Modeling - Volume Part I
Studying digital imagery of ancient paintings by mixtures of stochastic models

IEEE Transactions on Image Processing

Quantified Score

Hi-index	0.00

Visualization

Abstract

Knowledge of paintings domain includes a variety of sources such as essays, visual examples, ontologies of artistic concepts and user- provided annotations. This knowledge serves several purposes. First, it defines a wide range of concepts for annotation and flexible retrieval of paintings. Second, it serves to bootstrap auto-annotation and disambiguate the generated candidate labels. Third, the user-provided annotations serve to discover folksonomies of concepts and vernacular terms. In this paper, we propose a framework for paintings auto-annotation that incorporates user provided images and annotations, domain ontology and external knowledge sources. We utilize these sources of information to bootstrap and support the auto-annotation task, which is based on transductive inference mechanism that combines probabilistic clustering and multi-expert approach to generate labels. We further combine user-provided annotations with generated labels and domain ontology to disambiguate the concepts. In our experiments, we focus on the autoannotation of painting and demonstrate that the user-provided annotations significantly increase annotation accuracy.