Bridging Language Modeling and Divergence from Randomness Models: A Log-Logistic Model for IR

Authors:
Stéphane Clinchant;Eric Gaussier
Affiliations:
Xerox Research Center Europe, Meylan, France 38240 and LIG, Univ. Grenoble I, Grenoble cedex 9, Grenoble, France 38041;LIG, Univ. Grenoble I, Grenoble cedex 9, Grenoble, France 38041
Venue:
ICTIR '09 Proceedings of the 2nd International Conference on Theory of Information Retrieval: Advances in Information Retrieval Theory
Year:
2009

Citing 6
Cited 4

Pivoted document length normalization

SIGIR '96 Proceedings of the 19th annual international ACM SIGIR conference on Research and development in information retrieval
Probabilistic models of information retrieval based on measuring the divergence from randomness

ACM Transactions on Information Systems (TOIS)
A study of smoothing methods for language models applied to information retrieval

ACM Transactions on Information Systems (TOIS)
A formal study of information retrieval heuristics

Proceedings of the 27th annual international ACM SIGIR conference on Research and development in information retrieval
The BNB distribution for text modeling

ECIR'08 Proceedings of the IR research, 30th European conference on Advances in information retrieval
Improving term frequency normalization for multi-topical documents and application to language modeling approaches

ECIR'08 Proceedings of the IR research, 30th European conference on Advances in information retrieval

Statistical language models for information retrieval chengxiang zhai university of illinois at urbana champaign morgan & claypool (synthesis lectures on human language technologies, edited by graeme hirst), volume 1, 2008; xiii+125 pp, princeton, nj; paperbound, isbn 978-1-59829-590-0, $40.00; ebook, isbn 978-1-59829-591-7, $30.00 or by subscription

Computational Linguistics
Do IR models satisfy the TDC retrieval constraint

Proceedings of the 34th international ACM SIGIR conference on Research and development in Information Retrieval
A log-logistic model-based interpretation of TF normalization of BM25

ECIR'12 Proceedings of the 34th European conference on Advances in Information Retrieval
Pseudo test collections for training and tuning microblog rankers

Proceedings of the 36th international ACM SIGIR conference on Research and development in information retrieval

Quantified Score

Hi-index	0.00

Visualization

Abstract

We are interested in this paper in revisiting the Divergence from Randomness (DFR) approach to Information Retrieval (IR), so as to better understand the different contributions it relies on, and thus be able to simplify it. To do so, we first introduce an analytical characterization of heuristic retrieval constraints and review several DFR models wrt this characterization. This review shows that the first normalization principle of DFR is necessary to make the model compliant with retrieval constraints. We then show that the log-logistic distribution can be used to derive a simplified DFR model. Interestingly, this simplified model contains Language Models (LM) with Jelinek-Mercer smoothing. The relation we propose here is, to our knowledge, the first connection between the DFR and LM approaches. Lastly, we present experimental results obtained on several standard collections which validate the simplification and the model we propose.