A Graph Based Approach to Speaker Retrieval in Talk Show Videos with Transcript-Based Supervision

  • Authors:
  • Yina Han;Guizhong Liu;Hichem Sahbi;Gérard Chollet

  • Affiliations:
  • The School of Electronic and Information Engineering, Xi'an Jiaotong University, Xi'an, China 710049;The School of Electronic and Information Engineering, Xi'an Jiaotong University, Xi'an, China 710049;TELECOM-ParisTech, CNRS LTCI UMR 5141, Paris, France 75634;TELECOM-ParisTech, CNRS LTCI UMR 5141, Paris, France 75634

  • Venue:
  • PCM '09 Proceedings of the 10th Pacific Rim Conference on Multimedia: Advances in Multimedia Information Processing
  • Year:
  • 2009

Quantified Score

Hi-index 0.00

Visualization

Abstract

This paper proposes a graph based strategy to retrieve frames containing the queried speakers in talk show videos. Based on who is speaking and when information from the audio transcript, an initial audio-based step, that restricts the queried person to frames corresponding to when he/she is speaking, with a second step that analyzes visual features of shots is combined. Specifically, based on the production property of talk show video, (1) Shot based graph is constructed first. Then the densest sub-graph is returned as the final result. But instead of direct search (DS) of the densest part, (2) We model the intra node connection and inter node connection by a frame layer degree map to take into account the duration information within each shot node; (3)A graph partition strategy without restriction on the shape and the number of sub-graphs is proposed, in which shots containing the same person are more similar to each other. Experiments on one episode of the French talk show "Le Grand Echiquier" show more than 10% improvement to audio only method and more than 7.5% improvement to DS method on average.