Diverse Topic Phrase Extraction through Latent Semantic Analysis

  • Authors:
  • Jilin Chen;Jun Yan;Benyu Zhang;Qiang Yang;Zheng Chen

  • Affiliations:
  • University of Minnesota, USA;Microsoft Research Asia, China;Microsoft Research Asia, China;Hong Kong University of Science and Technology, Hong Kong;Microsoft Research Asia, China

  • Venue:
  • ICDM '06 Proceedings of the Sixth International Conference on Data Mining
  • Year:
  • 2006

Quantified Score

Hi-index 0.00

Visualization

Abstract

We propose a novel algorithm for extracting diverse topic phrases in order to provide summary for large corpora. Previous works often ignore the importance of diversity and thus extract phrases crowded on some hot topics while failing to cover other less obvious but important topics. We solve this problem through document re-weighting and phrase diversification by using latent semantic analysis (LSA). Experiments on various datasets show that our new algorithm can improve relevance as well as diversity over different topics for topic phrase extraction problems.