Learning to rank by aggregating expert preferences

  • Authors:
  • Maksims N. Volkovs;Hugo Larochelle;Richard S. Zemel

  • Affiliations:
  • University of Toronto, Toronto, ON, Canada;Université de Sherbrooke, Sherbrooke, PQ, Canada;University of Toronto, Toronto, ON, Canada

  • Venue:
  • Proceedings of the 21st ACM international conference on Information and knowledge management
  • Year:
  • 2012

Quantified Score

Hi-index 0.00

Visualization

Abstract

We present a general treatment of the problem of aggregating preferences from several experts into a consensus ranking, in the context where information about a target ranking is available. Specifically, we describe how such problems can be converted into a standard learning-to-rank one on which existing learning solutions can be invoked. This transformation allows us to optimize the aggregating function for any target IR metric, such as Normalized Discounted Cumulative Gain, or Expected Reciprocal Rank. When applied to crowdsourcing and meta-search benchmarks, our new algorithm improves on state-of-the-art preference aggregation methods.