Information preservation in static index pruning

Authors:
Ruey-Cheng Chen;Chia-Jung Lee;Chiung-Min Tsai;Jieh Hsiang
Affiliations:
National Taiwan University, Taipei, Taiwan Roc;University of Massachusetts, Amherst, MA, USA;National Taiwan University, Taipei, Taiwan Roc;National Taiwan University, Taipei, Taiwan Roc
Venue:
Proceedings of the 21st ACM international conference on Information and knowledge management
Year:
2012

Citing 6
Cited 2

The probability ranking principle in IR

Readings in information retrieval
Static index pruning for information retrieval systems

Proceedings of the 24th annual international ACM SIGIR conference on Research and development in information retrieval
A document-centric approach to static index pruning in text retrieval systems

CIKM '06 Proceedings of the 15th ACM international conference on Information and knowledge management
Entropy-Based Static Index Pruning

ECIR '09 Proceedings of the 31th European Conference on IR Research on Advances in Information Retrieval
Probabilistic static pruning of inverted files

ACM Transactions on Information Systems (TOIS)
Static pruning of terms in inverted files

ECIR'07 Proceedings of the 29th European conference on IR research

An information-theoretic account of static index pruning

Proceedings of the 36th international ACM SIGIR conference on Research and development in information retrieval
A Fast Static Index Pruning Algorithm

Proceedings of the Second International Conference on Innovative Computing and Cloud Computing

Quantified Score

Hi-index	0.00

Visualization

Abstract

We develop a new static index pruning criterion based on the notion of information preservation. This idea is motivated by the fact that model degeneration, as does static index pruning, inevitably reduces the predictive power of the resulting model. We model this loss in predictive power using conditional entropy and show that the decision in static index pruning can therefore be optimized to preserve information as much as possible. We evaluated the proposed approach on three different test corpora, and the result shows that our approach is comparable in retrieval performance to state-of-the-art methods. When efficiency is of concern, our method has some advantages over the reference methods and is therefore suggested in Web retrieval settings.