On the evaluation of entity profiles

Authors:
Maarten De Rijke;Krisztian Balog;Toine Bogers;Antal Van Den Bosch
Affiliations:
ISLA, University of Amsterdam, The Netherlands;ISLA, University of Amsterdam, The Netherlands;IIIA, Royal School of Library & Information Science, Denmark;ILK, Tilburg Centre for Creative Computing, Tilburg University, The Netherlands
Venue:
CLEF'10 Proceedings of the 2010 international conference on Multilingual and multimodal information access evaluation: cross-language evaluation forum
Year:
2010

Citing 10
Cited 1

Cumulated gain-based evaluation of IR techniques

ACM Transactions on Information Systems (TOIS)
Will pyramids built of nuggets topple over?

HLT-NAACL '06 Proceedings of the main conference on Human Language Technology Conference of the North American Chapter of the Association of Computational Linguistics
Broad expertise retrieval in sparse data environments

SIGIR '07 Proceedings of the 30th annual international ACM SIGIR conference on Research and development in information retrieval
The relationship between IR effectiveness measures and user satisfaction

SIGIR '07 Proceedings of the 30th annual international ACM SIGIR conference on Research and development in information retrieval
Novelty and diversity in information retrieval evaluation

Proceedings of the 31st annual international ACM SIGIR conference on Research and development in information retrieval
Overview of the WiQA Task at CLEF 2006

Evaluation of Multilingual and Multi-modal Information Retrieval
A language modeling framework for expert finding

Information Processing and Management: an International Journal
Determining expert profiles (with an application to expert finding)

IJCAI'07 Proceedings of the 20th international joint conference on Artifical intelligence
The impact of document structure on keyphrase extraction

Proceedings of the 18th ACM conference on Information and knowledge management
Learning Semantic Query Suggestions

ISWC '09 Proceedings of the 8th International Semantic Web Conference

Expertise Retrieval

Foundations and Trends in Information Retrieval

Quantified Score

Hi-index	0.00

Visualization

Abstract

Entity profiling is the task of identifying and ranking descriptions of a given entity. The task may be viewed as one where the descriptions being sought are terms that need to be selected from a knowledge source (such as an ontology or thesaurus). In this case, entity profiling systems can be assessed by means of precision and recall values of the descriptive terms produced. However, recent evidence suggests thatmore sophisticated metrics are needed that go beyond mere lexical matching of system-produced descriptors against a ground truth, allowing for graded relevance and rewarding diversity in the list of descriptors returned. In this note, we motivate and propose such a metric.