Multi-scale TextTiling for automatic story segmentation in Chinese broadcast news

  • Authors:

  • Lei Xie;Jia Zeng;Wei Feng

  • Affiliations:

  • Audio, Speech & Language Processing Group, School of Computer Science, Northwestern Polytechnical University, Xi'an, China;Department of Electronic Engineering, City University of Hong Kong, Hong Kong SAR;School of Creative Media, City University of Hong Kong, Hong Kong SAR

  • Venue:
  • AIRS'08 Proceedings of the 4th Asia information retrieval conference on Information retrieval technology
  • Year:
  • 2008

Quantified Score

Hi-index 0.00

Visualization

Abstract

This paper applies Chinese subword representations, namely character and syllable n-grams, into the TextTiling-based automatic story segmentation of Chinese broadcast news. We show the robustness of Chinese subwords against speech recognition errors, out-of-vocabulary (OOV) words and versatility in word segmentation in lexical matching on errorful Chinese speech recognition transcripts. We propose a multi-scale TextTiling approach that integrates both the specificity of words and the robustness of subwords in lexical similarity measure for story boundary identification. Experiments on the TDT2 Mandarin corpus show that subword bigrams achieve the best performance among all scales with relative f -measure improvement of 8.84% (character bigram) and 7.11% (syllable bigram) over words. Multi-scale fusion of subword bigrams with words can bring further improvement. It is promising that the integration of syllable bigram with syllable sequence of word achieves an f -measure gain of 2.66% over the syllable bigram alone.