Inferring semantics from textual information in multimedia retrieval

Authors:
Mats Sjöberg;Jorma Laaksonen;Timo Honkela;Matti Pöllä
Affiliations:
Adaptive Informatics Research Centre, Helsinki University of Technology, P.O. Box 5400, 02015 TKK, Finland;Adaptive Informatics Research Centre, Helsinki University of Technology, P.O. Box 5400, 02015 TKK, Finland;Adaptive Informatics Research Centre, Helsinki University of Technology, P.O. Box 5400, 02015 TKK, Finland;Adaptive Informatics Research Centre, Helsinki University of Technology, P.O. Box 5400, 02015 TKK, Finland
Venue:
Neurocomputing
Year:
2008

Citing 19
Cited 0

Invariant Image Recognition by Zernike Moments

IEEE Transactions on Pattern Analysis and Machine Intelligence
Word association norms, mutual information, and lexicography

Computational Linguistics
The symbol grounding problem

CNLS '89 Proceedings of the ninth annual international conference of the Center for Nonlinear Studies on Self-organizing, Collective, and Cooperative Phenomena in Natural and Artificial Computing Networks on Emergent computation
Comparison of parametric representations for monosyllabic word recognition in continuously spoken sentences

Readings in speech recognition
A self-organizing semantic map for information retrieval

SIGIR '91 Proceedings of the 14th annual international ACM SIGIR conference on Research and development in information retrieval
Dimensions of meaning

Proceedings of the 1992 ACM/IEEE conference on Supercomputing
Subsymbolic natural language processing: an integrated model of scripts, lexicon, and memory

Subsymbolic natural language processing: an integrated model of scripts, lexicon, and memory
Generating and evaluating domain-oriented multi-word terms from texts

Information Processing and Management: an International Journal
Foundations of statistical natural language processing

Foundations of statistical natural language processing
A vector space model for automatic indexing

Communications of the ACM
Self-Organizing Maps

Self-Organizing Maps
Introduction to Modern Information Retrieval

Introduction to Modern Information Retrieval
Matching words and pictures

The Journal of Machine Learning Research
On image auto-annotation with latent space models

MULTIMEDIA '03 Proceedings of the eleventh ACM international conference on Multimedia
Multi-level annotation of natural scenes using dominant image components and semantic concepts

Proceedings of the 12th annual ACM international conference on Multimedia
GCap: Graph-based Automatic Image Captioning

CVPRW '04 Proceedings of the 2004 Conference on Computer Vision and Pattern Recognition Workshop (CVPRW'04) Volume 9 - Volume 09
Retrieval of multimedia objects by combining semantic information from visual and textual descriptors

ICANN'06 Proceedings of the 16th international conference on Artificial Neural Networks - Volume Part II
Self organization of a massive document collection

IEEE Transactions on Neural Networks
PicSOM-self-organizing image retrieval with MPEG-7 content descriptors

IEEE Transactions on Neural Networks

Quantified Score

Hi-index	0.01

Visualization

Abstract

We propose a method for inferring semantic information from textual data in content-based multimedia retrieval. Training examples of images and videos belonging to a specific semantic class are associated with their low-level visual and aural descriptors augmented with textual features such as frequencies of significant words. A fuzzy mapping of a semantic class in the training set to a class of similar objects in the test set is created by using Self-Organizing Maps (SOMs) trained from the low-level descriptors. Experiments with two databases and different textual features show promising results, indicating the usefulness of the approach in bridging the gap from low-level visual features to semantic concepts.