Text segmentation via topic modeling: an analytical study

  • Authors:
  • Hemant Misra;François Yvon;Joemon M. Jose;Olivier Cappe

  • Affiliations:
  • University of Glasgow, Glasgow, United Kingdom;University Paris-Sud and LIMSI-CNRS, Orsay, France;University of Glasgow, Glasgow, United Kingdom;TELECOM ParisTech and LTCI/CNRS, Paris, France

  • Venue:
  • Proceedings of the 18th ACM conference on Information and knowledge management
  • Year:
  • 2009

Quantified Score

Hi-index 0.00

Visualization

Abstract

In this paper, the task of text segmentation is approached from a topic modeling perspective. We investigate the use of latent Dirichlet allocation (LDA) topic model to segment a text into semantically coherent segments. A major benefit of the proposed approach is that along with the segment boundaries, it outputs the topic distribution associated with each segment. This information is of potential use in applications like segment retrieval and discourse analysis. The new approach outperforms a standard baseline method and yields significantly better performance than most of the available unsupervised methods on a benchmark dataset.