A system for the semantic multimodal analysis of news audio-visual content

Authors:
Vasileios Mezaris;Spyros Gidaros;Walter Kasper;Jörg Steffen;Roeland Ordelman;Marijn Huijbregts;Franciska de Jong;Ioannis Kompatsiaris;Michael G. Strintzis
Affiliations:
Centre for Research and Technology Hellas, Informatics and Telematics Institute, Thermi, Greece;Centre for Research and Technology Hellas, Informatics and Telematics Institute, Thermi, Greece;Language Technology Laboratory, DFKI GmbH, Saarbrucken, Germany;Language Technology Laboratory, DFKI GmbH, Saarbrucken, Germany;Department of Computer Science, Human Media Interaction, University of Twente, Enschede, The Netherlands;Department of Computer Science, Human Media Interaction, University of Twente, Enschede, The Netherlands and Centre for Language and Speech Technology, Radboud University Nijmegen, Nijmegen, The N ...;Department of Computer Science, Human Media Interaction, University of Twente, Enschede, The Netherlands;Centre for Research and Technology Hellas, Informatics and Telematics Institute, Thermi, Greece;Centre for Research and Technology Hellas, Informatics and Telematics Institute, Thermi, Greece and Department of Electrical and Computer Engineering, Aristotle University of Thessaloniki, Thessal ...
Venue:
EURASIP Journal on Advances in Signal Processing
Year:
2010

Citing 19
Cited 2

The logic of typed feature structures

The logic of typed feature structures
Models for metasearch

Proceedings of the 24th annual international ACM SIGIR conference on Research and development in information retrieval
The Holy Grail of Content-Based Media Analysis

IEEE MultiMedia
News video classification using SVM-based multimodal classifiers and combination strategies

Proceedings of the tenth ACM international conference on Multimedia
Robust Real-Time Face Detection

International Journal of Computer Vision
Multi-modal classification in digital news libraries

Proceedings of the 4th ACM/IEEE-CS joint conference on Digital libraries
Histograms of Oriented Gradients for Human Detection

CVPR '05 Proceedings of the 2005 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'05) - Volume 1 - Volume 01
Evaluating the application of semantic inferencing rules to image annotation

Proceedings of the 3rd international conference on Knowledge capture
Supervised rank aggregation

Proceedings of the 16th international conference on World Wide Web
Combining global and local information for knowledge-assisted image analysis and classification

EURASIP Journal on Advances in Signal Processing
Integrating multi-modal content analysis and hyperbolic visualization for large-scale news video retrieval and exploration

Image Communication
A Novel Video Searching Model Based on Ontology Inference and Multimodal Information Fusion

ISCSCT '08 Proceedings of the 2008 International Symposium on Computer Science and Computational Technology - Volume 02
Adaptive multimodal fusion by uncertainty compensation with application to audiovisual speech recognition

IEEE Transactions on Audio, Speech, and Language Processing - Special issue on multimodal processing in speech-based interactions
Ontology-driven semantic video analysis using visual information objects

SAMT'07 Proceedings of the semantic and digital media technologies 2nd international conference on Semantic Multimedia
The AMI speaker diarization system for NIST RT06s meeting data

MLMI'06 Proceedings of the Third international conference on Machine Learning for Multimodal Interaction
The MPEG-7 visual standard for content description-an overview

IEEE Transactions on Circuits and Systems for Video Technology
Real-time shot change detection over online MPEG-2 video

IEEE Transactions on Circuits and Systems for Video Technology
Knowledge-assisted semantic video object detection

IEEE Transactions on Circuits and Systems for Video Technology
Multimedia Search Without Visual Analysis: The Value of Linguistic and Contextual Information

IEEE Transactions on Circuits and Systems for Video Technology

Exploiting information extraction techniques for automatic semantic video indexing with an application to Turkish news videos

Knowledge-Based Systems
Intent and its discontents: the user at the wheel of the online video search engine

Proceedings of the 20th ACM international conference on Multimedia

Quantified Score

Hi-index	0.00

Visualization

Abstract

News-related content is nowadays among the most popular types of content for users in everyday applications. Although the generation and distribution of news content has become commonplace, due to the availability of inexpensive media capturing devices and the development of media sharing services targeting both professional and user-generated news content, the automatic analysis and annotation that is required for supporting intelligent search and delivery of this content remains an open issue. In this paper, a complete architecture for knowledge-assisted multimodal analysis of news-related multimedia content is presented, along with its constituent components. The proposed analysis architecture employs state-of-the-art methods for the analysis of each individual modality (visual, audio, text) separately and proposes a novel fusion technique based on the particular characteristics of news-related content for the combination of the individual modality analysis results. Experimental results on news broadcast video illustrate the usefulness of the proposed techniques in the automatic generation of semantic annotations.