Benchmarking domain-specific expert search using workshop program committees

Authors:
Georgeta Bordea;Toine Bogers;Paul Buitelaar
Affiliations:
Digital Enterprise Research Institute, National University of Ireland, Galway, Galway, Ireland;University of Copenhagen, Copenhagen, Denmark;Digital Enterprise Research Institute, National University of Ireland, Galway, Galway, Ireland
Venue:
Proceedings of the 2013 workshop on Computational scientometrics: theory & applications
Year:
2013

Citing 11
Cited 0

Automating the assignment of submitted manuscripts to reviewers

SIGIR '92 Proceedings of the 15th annual international ACM SIGIR conference on Research and development in information retrieval
Expertise identification using email communications

CIKM '03 Proceedings of the twelfth international conference on Information and knowledge management
Broad expertise retrieval in sparse data environments

SIGIR '07 Proceedings of the 30th annual international ACM SIGIR conference on Research and development in information retrieval
Expertise modeling for matching papers with reviewers

Proceedings of the 13th ACM SIGKDD international conference on Knowledge discovery and data mining
ArnetMiner: extraction and mining of academic social networks

Proceedings of the 14th ACM SIGKDD international conference on Knowledge discovery and data mining
An algorithm to determine peer-reviewers

Proceedings of the 17th ACM conference on Information and knowledge management
Formal Models for Expert Finding on DBLP Bibliography Data

ICDM '08 Proceedings of the 2008 Eighth IEEE International Conference on Data Mining
Contextual factors for finding similar experts

Journal of the American Society for Information Science and Technology
A user-oriented model for expert finding

ECIR'11 Proceedings of the 33rd European conference on Advances in information retrieval
Linked data metrics for flexible expert search on the open web

ESWC'11 Proceedings of the 8th extended semantic web conference on The semantic web: research and applications - Volume Part I
Automatic topics identification for reviewer assignment

IEA/AIE'06 Proceedings of the 19th international conference on Advances in Applied Artificial Intelligence: industrial, Engineering and Other Applications of Applied Intelligent Systems

Quantified Score

Hi-index	0.00

Visualization

Abstract

Traditionally, relevance assessments for expert search have been gathered through self-assessment or based on the opinions of co-workers. We introduce three benchmark datasets for expert search that use conference workshops for relevance assessment. Our data sets cover entire research domains as opposed to single institutions. In addition, they provide a larger number of topic-person associations and allow a more objective and fine-grained evaluation of expertise than existing data sets do. We present and discuss baseline results for a language modelling and a topic-centric approach to expert search. We find that the topic-centric approach achieves the best results on domain-specific datasets.