Mining pure high-order word associations via information geometry for information retrieval

  • Authors:

  • Yuexian Hou;Xiaozhao Zhao;Dawei Song;Wenjie Li

  • Affiliations:

  • Tianjin University, China;Tianjin University, China;Tianjin University and Open University, China;Hong Kong Polytechnic University, Hong Kong

  • Venue:
  • ACM Transactions on Information Systems (TOIS)
  • Year:
  • 2013

Quantified Score

Hi-index 0.00

Visualization

Abstract

The classical bag-of-word models for information retrieval (IR) fail to capture contextual associations between words. In this article, we propose to investigate pure high-order dependence among a number of words forming an unseparable semantic entity, that is, the high-order dependence that cannot be reduced to the random coincidence of lower-order dependencies. We believe that identifying these pure high-order dependence patterns would lead to a better representation of documents and novel retrieval models. Specifically, two formal definitions of pure dependence—unconditional pure dependence (UPD) and conditional pure dependence (CPD)—are defined. The exact decision on UPD and CPD, however, is NP-hard in general. We hence derive and prove the sufficient criteria that entail UPD and CPD, within the well-principled information geometry (IG) framework, leading to a more feasible UPD/CPD identification procedure. We further develop novel methods for extracting word patterns with pure high-order dependence. Our methods are applied to and extensively evaluated on three typical IR tasks: text classification and text retrieval without and with query expansion.