Tools for Browsing a TV Situation Comedy Based on Content Specific Attributes

Authors:
Joshua S. Wachman;Rosalind W. Picard
Affiliations:
The Perceptual Computing Group of the MIT Media Laboratory, 20 Ames Street, Cambridge, MA 02139, USA. wachman@watchpointmedia.com;The Perceptual Computing Group of the MIT Media Laboratory, 20 Ames Street, Cambridge, MA 02139, USA. picard@media.mit.edu
Venue:
Multimedia Tools and Applications
Year:
2001

Citing 7
Cited 7

Algorithms for clustering data

Algorithms for clustering data
Color indexing

International Journal of Computer Vision
Automatic partitioning of full-motion video

Multimedia Systems
Vision texture for annotation

Multimedia Systems - Special issue on content-based retrieval
Video parsing and browsing using compressed data

Multimedia Tools and Applications
Interactive Learning with a "Society of Models"

CVPR '96 Proceedings of the 1996 Conference on Computer Vision and Pattern Recognition (CVPR '96)
Efficient matching and clustering of video shots

ICIP '95 Proceedings of the 1995 International Conference on Image Processing (Vol. 1)-Volume 1 - Volume 1

Multimodal Person Identification in Movies

CIVR '02 Proceedings of the International Conference on Image and Video Retrieval
Interactive Adaptive Movie Annotation

IEEE MultiMedia
Experiential meeting system

ETP '03 Proceedings of the 2003 ACM SIGMM workshop on Experiential telepresence
Components and systems for interactive video indexing

ICME '03 Proceedings of the 2003 International Conference on Multimedia and Expo - Volume 2
A framework for aligning and indexing movies with their script

ICME '03 Proceedings of the 2003 International Conference on Multimedia and Expo - Volume 2
Technologies That Make You Smile: Adding Humor to Text-Based Applications

IEEE Intelligent Systems
SeeNSearch: A context directed search facilitator for home entertainment devices

Pervasive and Mobile Computing

Quantified Score

Hi-index	0.00

Visualization

Abstract

This paper presents general purpose video analysis and annotation tools, which combine high-level and low-level information, and which learn through user interaction and feedback. The use of these tools is illustrated through the construction of two video browsers, which allow a user to fast forward (or rewind) to frames, shots, or scenes containing a particular character, characters, or other labeled content. The two browsers developed in this work are: (1) a basic video browser, which exploits relations between high-level scripting information and closed captions, and (2) an advanced video browser, which augments the basic browser with annotations gained from applying machine learning. The learner helps the system adapt to different peoples' labelings by accepting positive and negative examples of labeled content from a user, and relating these to low-level color and texture features extracted from the digitized video. This learning happens interactively, and is used to infer labels on data the user has not yet seen. The labeled data may then be browsed or retrieved from the database in real time.An evaluation of the learning performance shows that a combination of low-level color signal features outperforms several other combinations of signal features in learning character labels in an episode of the TV situation comedy, Seinfeld. We discuss several issues that arise in the combination of low-level and high-level information, and illustrate solutions to these issues within the context of browsing television sitcoms.