The opposite of smoothing: a language model approach to ranking query-specific document clusters

  • Authors:
  • Oren Kurland;Eyal Krikon

  • Affiliations:
  • Faculty of Industrial Engineering and Management Technion, Israel Institute of Technology;Faculty of Industrial Engineering and Management Technion, Israel Institute of Technology

  • Venue:
  • Journal of Artificial Intelligence Research
  • Year:
  • 2011

Quantified Score

Hi-index 0.00

Visualization

Abstract

Exploiting information induced from (query-specific) clustering of top-retrieved documents has long been proposed as a means for improving precision at the very top ranks of the returned results. We present a novel language model approach to ranking query-specific clusters by the presumed percentage of relevant documents that they contain. While most previous cluster ranking approaches focus on the cluster as a whole, our model utilizes also information induced from documents associated with the cluster. Our model substantially outperforms previous approaches for identifying clusters containing a high relevant-document percentage. Furthermore, using the model to produce document ranking yields precision-at-top-ranks performance that is consistently better than that of the initial ranking upon which clustering is performed. The performance also favorably compares with that of a state-of-the-art pseudo-feedback-based retrieval method.