Evaluating scalability in information retrieval with multigraded relevance

Authors:
Amélie Imafouo;Michel Beigbeder
Affiliations:
Ecole Nationale Supérieure des Mines of Saint-Etienne, Saint-Etienne, France;Ecole Nationale Supérieure des Mines of Saint-Etienne, Saint-Etienne, France
Venue:
AIRS'06 Proceedings of the Third Asia conference on Information Retrieval Technology
Year:
2006

Citing 10
Cited 0

A re-examination of relevance: toward a dynamic, situational definition

Information Processing and Management: an International Journal
User-defined relevance criteria: an exploratory study

Journal of the American Society for Information Science - Special issue: relevance research
Towards the identification of the optimal number of relevance categories

Journal of the American Society for Information Science
From highly relevant to not relevant: examining different regions of relevance

Information Processing and Management: an International Journal
IR evaluation methods for retrieving highly relevant documents

SIGIR '00 Proceedings of the 23rd annual international ACM SIGIR conference on Research and development in information retrieval
Dimensions of relevance

Information Processing and Management: an International Journal
Evaluation by highly relevant documents

Proceedings of the 24th annual international ACM SIGIR conference on Research and development in information retrieval
Using graded relevance assessments in IR evaluation

Journal of the American Society for Information Science and Technology
Average gain ratio: a simple retrieval performance measure for evaluation with multiple relevance levels

Proceedings of the 26th annual international ACM SIGIR conference on Research and development in informaion retrieval
Binary and graded relevance in IR evaluations-Comparison of the effects on ranking of IR systems

Information Processing and Management: an International Journal

Quantified Score

Hi-index	0.00

Visualization

Abstract

For the user’s point of view, in large environments, it can be desirable to have Information Retrieval Systems (IRS) that retrieve documents according to their relevance levels. Relevance levels have been studied in some previous Information Retrieval (IR) works while some others (few) IR research works tackled the questions of IRS effectiveness and collections size. These latter works used standard IR measures on collections of increasing size to analyze IRS effectiveness scalability. In this work, we bring together these two issues in IR (multigraded relevance and scalability) by designing some new metrics for evaluating the ability of IRS to rank documents according to their relevance levels when collection size increases.