Managing short postings lists

Authors:
Andrew Trotman;Xiang-Fei Jia;Matt Crane
Affiliations:
University of Otago, Dunedin, New Zealand;University of Otago, Dunedin, New Zealand;University of Otago, Dunedin, New Zealand
Venue:
Proceedings of the 18th Australasian Document Computing Symposium
Year:
2013

Citing 11
Cited 0

Memory efficient ranking

Information Processing and Management: an International Journal - Special issue: data compression
Query evaluation: strategies and optimizations

Information Processing and Management: an International Journal
Filtered document retrieval with frequency-sorted indexes

Journal of the American Society for Information Science
Vector-space ranking with effective early termination

Proceedings of the 24th annual international ACM SIGIR conference on Research and development in information retrieval
Compressing Inverted Files

Information Retrieval
Efficient query evaluation using a two-level retrieval process

CIKM '03 Proceedings of the twelfth international conference on Information and knowledge management
Inverted Index Compression Using Word-Aligned Binary Codes

Information Retrieval
Super-Scalar RAM-CPU Cache Compression

ICDE '06 Proceedings of the 22nd International Conference on Data Engineering
Improved Word-Aligned Binary Compression for Text Indexing

IEEE Transactions on Knowledge and Data Engineering
Sigma encoded inverted files

Proceedings of the sixteenth ACM conference on Conference on information and knowledge management
Application-Specific Disk I/O Optimisation for a Search Engine

PDCAT '08 Proceedings of the 2008 Ninth International Conference on Parallel and Distributed Computing, Applications and Technologies

Quantified Score

Hi-index	0.00

Visualization

Abstract

Previous work has examined space saving and throughput increasing techniques for long postings lists in an inverted file search engine. In this contribution we show that highly sporadic terms (terms that occur in 1 or 2 documents) are a high proportion of the unique terms in the collection and that these terms are seen in queries. The previously known space saving method of storing their short postings lists in the vocabulary is compared to storing in the postings file. We quantify the saving as about 6.5%, with no loss in precision, and suggest the adoption of this technique.