A study on novelty evaluation in biomedical information retrieval

Authors:
Xiangdong An;Nick Cercone;Hai Wang;Zheng Ye
Affiliations:
York University, Toronto, ON, Canada;York University, Toronto, ON, Canada;Saint Mary's University, Halifax, NS, Canada;York University, Toronto, ON, Canada
Venue:
SPIRE'12 Proceedings of the 19th international conference on String Processing and Information Retrieval
Year:
2012

Citing 8
Cited 0

The use of MMR, diversity-based reranking for reordering documents and producing summaries

Proceedings of the 21st annual international ACM SIGIR conference on Research and development in information retrieval
Novelty and redundancy detection in adaptive filtering

SIGIR '02 Proceedings of the 25th annual international ACM SIGIR conference on Research and development in information retrieval
Cumulated gain-based evaluation of IR techniques

ACM Transactions on Information Systems (TOIS)
Beyond independent relevance: methods and evaluation metrics for subtopic retrieval

Proceedings of the 26th annual international ACM SIGIR conference on Research and development in informaion retrieval
Improving web search results using affinity graph

Proceedings of the 28th annual international ACM SIGIR conference on Research and development in information retrieval
Statistical precision of information retrieval evaluation

SIGIR '06 Proceedings of the 29th annual international ACM SIGIR conference on Research and development in information retrieval
Novelty and diversity in information retrieval evaluation

Proceedings of the 31st annual international ACM SIGIR conference on Research and development in information retrieval
Structural relevance: a common basis for the evaluation of structured document retrieval

Proceedings of the 17th ACM conference on Information and knowledge management

Quantified Score

Hi-index	0.00

Visualization

Abstract

In novelty information retrieval, we expect that novel passages are ranked higher than redundant ones and relevant ones higher than irrelevant ones. Accordingly, we desire an evaluation algorithm that would respect such expectations. In TREC 2006 & 2007, a novelty performance measure, called the aspect-based mean average precision (MAP), was introduced to the Genomics Track to rank the novelty of the medical passages. In this paper, we demonstrate that this measure may not necessarily yeild a higher score for the rankings that honor above expectations better. We propose an improved measure to reflect such expectations more precisely, and present some supporting evidences.