NLP for indexing and retrieval of captioned photographs

Authors:
Katerina Pastra;Horacio Saggion;Yorick Wilks
Affiliations:
University of Sheffield, England - UK;University of Sheffield, England - UK;University of Sheffield, England - UK
Venue:
EACL '03 Proceedings of the tenth conference on European chapter of the Association for Computational Linguistics - Volume 2
Year:
2003

Citing 2
Cited 1

Natural-language retrieval of images based on descriptive captions

ACM Transactions on Information Systems (TOIS)
Intelligent Indexing of Crime Scene Photographs

IEEE Intelligent Systems

Composing simple image descriptions using web-scale n-grams

CoNLL '11 Proceedings of the Fifteenth Conference on Computational Natural Language Learning

Quantified Score

Hi-index	0.00

Visualization

Abstract

We present a text-based approach for the automatic indexing and retrieval of digital photographs taken at crime scenes. Our research prototype, SOCIS, goes beyond keyword-based approaches and methods that extract syntactic relations from captions; it relies on advanced Natural Language Processing techniques in order to extract relational facts. These relational facts consist of a "pragmatic relation" and the entities this relation connects (triples of the form: ARG1-REL- ARG2). In SOCIS, the triples are used as complex image indexing terms; however, the extraction mechanism is used not only for indexing purposes but also for image retrieval using free text queries. The retrieval mechanism computes similarity scores between query-triples and indexing-triples making use of a domain-specific ontology.