A Modification of LambdaMART to Handle Noisy Crowdsourced Assessments

Authors:
Pavel Metrikov;Jie Wu;Jesse Anderton;Virgil Pavlu;Javed A. Aslam
Affiliations:
College of Computer and Information Science, Northeastern University, Boston, MA, USA;College of Computer and Information Science, Northeastern University, Boston, MA, USA;College of Computer and Information Science, Northeastern University, Boston, MA, USA;College of Computer and Information Science, Northeastern University, Boston, MA, USA;College of Computer and Information Science, Northeastern University, Boston, MA, USA
Venue:
Proceedings of the 2013 Conference on the Theory of Information Retrieval
Year:
2013

Citing 2
Cited 0

Bagging gradient-boosted trees for high precision, low variance ranking models

Proceedings of the 34th international ACM SIGIR conference on Research and development in Information Retrieval
On aggregating labels from multiple crowd workers to infer relevance of documents

ECIR'12 Proceedings of the 34th European conference on Advances in Information Retrieval

Quantified Score

Hi-index	0.00

Visualization

Abstract

We consider noisy crowdsourced assessments and their impact on learning-to-rank algorithms. Starting with EM-weighted assessments, we modify LambdaMART in order to use smoothed probabilistic preferences over pairs of documents, directly as input to the ranking algorithm.