SeLeCT: a lexical cohesion based news story segmentation system

  • Authors:
  • Nicola Stokes;Joe Carthy;Alan F. Smeaton

  • Affiliations:
  • Department of Computer Science, University College Dublin, Ireland;Department of Computer Science, University College Dublin, Ireland;School for Computer Applications and Centre for Digital Video Processing, Dublin City University, Ireland

  • Venue:
  • AI Communications - STAIRS 2002
  • Year:
  • 2004

Quantified Score

Hi-index 0.00

Visualization

Abstract

In this paper we compare the performance of three distinct approaches to lexical cohesion based text segmentation. Most work in this area has focused on the discovery of textual units that discuss subtopic structure within documents. In contrast our segmentation task requires the discovery of topical units of text i.e., distinct news stories from broadcast news programmes. Our approach to news story segmentation (the SeLeCT system) is based on an analysis of lexical cohesive strength between textual units using a linguistic technique called lexical chaining. We evaluate the relative performance of SeLeCT with respect to two other cohesion based segmenters: TextTiling and C99. Using a recently introduced evaluation metric WindowDiff, we contrast the segmentation accuracy of each system on both "spoken" (CNN news transcripts) and "written" (Reuters newswire) news story test sets extracted from the TDT1 corpus.