IR evaluation methods for retrieving highly relevant documents
SIGIR '00 Proceedings of the 23rd annual international ACM SIGIR conference on Research and development in information retrieval
Optimizing search engines using clickthrough data
Proceedings of the eighth ACM SIGKDD international conference on Knowledge discovery and data mining
Large Margin Methods for Structured and Interdependent Output Variables
The Journal of Machine Learning Research
Learning to rank using gradient descent
ICML '05 Proceedings of the 22nd international conference on Machine learning
A support vector method for multivariate performance measures
ICML '05 Proceedings of the 22nd international conference on Machine learning
Training linear SVMs in linear time
Proceedings of the 12th ACM SIGKDD international conference on Knowledge discovery and data mining
Data Mining: Practical Machine Learning Tools and Techniques, Second Edition (Morgan Kaufmann Series in Data Management Systems)
Solving multiclass support vector machines with LaRank
Proceedings of the 24th international conference on Machine learning
Learning to rank: from pairwise approach to listwise approach
Proceedings of the 24th international conference on Machine learning
A support vector method for optimizing average precision
SIGIR '07 Proceedings of the 30th annual international ACM SIGIR conference on Research and development in information retrieval
FRank: a ranking method with fidelity loss
SIGIR '07 Proceedings of the 30th annual international ACM SIGIR conference on Research and development in information retrieval
SoftRank: optimizing non-smooth rank metrics
WSDM '08 Proceedings of the 2008 International Conference on Web Search and Data Mining
Interactively optimizing information retrieval systems as a dueling bandits problem
ICML '09 Proceedings of the 26th Annual International Conference on Machine Learning
Robust sparse rank learning for non-smooth ranking measures
Proceedings of the 32nd international ACM SIGIR conference on Research and development in information retrieval
Predicting structured objects with support vector machines
Communications of the ACM - Scratch Programming for All
Learning to Rank for Information Retrieval
Foundations and Trends in Information Retrieval
Gradient descent optimization of smoothed information retrieval metrics
Information Retrieval
Learning dense models of query similarity from user click logs
HLT '10 Human Language Technologies: The 2010 Annual Conference of the North American Chapter of the Association for Computational Linguistics
Learning to rank with multiple objective functions
Proceedings of the 20th international conference on World wide web
Learning to rank for why-question answering
Information Retrieval
Learning multiple metrics for ranking
Frontiers of Computer Science in China
Maximum margin ranking algorithms for information retrieval
ECIR'2010 Proceedings of the 32nd European conference on Advances in Information Retrieval
Active learning of combinatorial features for interactive optimization
LION'05 Proceedings of the 5th international conference on Learning and Intelligent Optimization
Learning to rank by optimizing expected reciprocal rank
AIRS'11 Proceedings of the 7th Asia conference on Information Retrieval Technology
CLiMF: learning to maximize reciprocal rank with collaborative less-is-more filtering
Proceedings of the sixth ACM conference on Recommender systems
A low rank structural large margin method for cross-modal ranking
Proceedings of the 36th international ACM SIGIR conference on Research and development in information retrieval
Direct optimization of ranking measures for learning to rank models
Proceedings of the 19th ACM SIGKDD international conference on Knowledge discovery and data mining
Cross-media semantic representation via bi-directional learning to rank
Proceedings of the 21st ACM international conference on Multimedia
Proceedings of the 23rd international conference on World wide web
Hi-index | 0.00 |
Learning to rank from relevance judgment is an active research area. Itemwise score regression, pairwise preference satisfaction, and listwise structured learning are the major techniques in use. Listwise structured learning has been applied recently to optimize important non-decomposable ranking criteria like AUC (area under ROC curve) and MAP (mean average precision). We propose new, almost-linear-time algorithms to optimize for two other criteria widely used to evaluate search systems: MRR (mean reciprocal rank) and NDCG (normalized discounted cumulative gain) in the max-margin structured learning framework. We also demonstrate that, for different ranking criteria, one may need to use different feature maps. Search applications should not be optimized in favor of a single criterion, because they need to cater to a variety of queries. E.g., MRR is best for navigational queries, while NDCG is best for informational queries. A key contribution of this paper is to fold multiple ranking loss functions into a multi-criteria max-margin optimization. The result is a single, robust ranking model that is close to the best accuracy of learners trained on individual criteria. In fact, experiments over the popular LETOR and TREC data sets show that, contrary to conventional wisdom, a test criterion is often not best served by training with the same individual criterion.