Multi-modal information retrieval from broadcast video using OCR and speech recognition

Authors:
Alexander G. Hauptmann;Rong Jin;Tobun Dorbin Ng
Affiliations:
Carnegie Mellon University, Pittsburgh, PA;Carnegie Mellon University, Pittsburgh, PA;Carnegie Mellon University, Pittsburgh, PA
Venue:
Proceedings of the 2nd ACM/IEEE-CS joint conference on Digital libraries
Year:
2002

Citing 2
Cited 3

Content-Based Image Retrieval at the End of the Early Years

IEEE Transactions on Pattern Analysis and Machine Intelligence
Video OCR for Digital News Archive

CAIVD '98 Proceedings of the 1998 International Workshop on Content-Based Access of Image and Video Databases (CAIVD '98)

Detection of video sequences using compact signatures

ACM Transactions on Information Systems (TOIS)
TalkMiner: a lecture webcast search engine

Proceedings of the international conference on Multimedia
Multi-modal solution for unconstrained news story retrieval

MMM'12 Proceedings of the 18th international conference on Advances in Multimedia Modeling

Quantified Score

Hi-index	0.00

Visualization

Abstract

We examine multi-modal information retrieval from broadcast video where text can be read on the screen through OCR and speech recognition can be performed on the audio track. OCR and speech recognition are compared on the 2001 TREC Video Retrieval evaluation corpus. Results show that OCR is more important that speech recognition for video retrieval. OCR retrieval can further improve through dictionary-based post-processing. We demonstrate how to utilize imperfect multi-modal metadata results to benefit multi-modal information retrieval.