Blogger, stick to your story: modeling topical noise in blogs with coherence measures

  • Authors:
  • Jiyin He;Wouter Weerkamp;Martha Larson;Maarten de Rijke

  • Affiliations:
  • University of Amsterdam, Amsterdam;University of Amsterdam, Amsterdam;University of Amsterdam, Amsterdam;University of Amsterdam, Amsterdam

  • Venue:
  • Proceedings of the second workshop on Analytics for noisy unstructured text data
  • Year:
  • 2008

Quantified Score

Hi-index 0.00

Visualization

Abstract

Topical noise in blogs arises when bloggers digress from the central topical thrust of their blogs. We introduce a method to explicitly incorporate a model of topical noise into a language modeling approach to the task of blog distillation. Topical noise is integrated into the model using a coherence score, which reflects the tightness of the topical structure of a blog. Tests performed on the TRECBlog06 corpus show that a naive integration of the coherence score as blog prior fails to achieve performance improvements. Instead, we develop a set of more sophisticated models in which the coherence score is weighted by a function of the blog retrieval score. The proposed models help improve effectiveness of our language modeling approach to the blog distillation task.