A decision-theoretic generalization of on-line learning and an application to boosting
Journal of Computer and System Sciences - Special issue: 26th annual ACM symposium on the theory of computing & STOC'94, May 23–25, 1994, and second annual Europe an conference on computational learning theory (EuroCOLT'95), March 13–15, 1995
Genetic programming: an introduction: on the automatic evolution of computer programs and its applications
A language modeling approach to information retrieval
Proceedings of the 21st annual international ACM SIGIR conference on Research and development in information retrieval
IR evaluation methods for retrieving highly relevant documents
SIGIR '00 Proceedings of the 23rd annual international ACM SIGIR conference on Research and development in information retrieval
Document language models, query models, and risk minimization for information retrieval
Proceedings of the 24th annual international ACM SIGIR conference on Research and development in information retrieval
Modern Information Retrieval
Optimizing search engines using clickthrough data
Proceedings of the eighth ACM SIGKDD international conference on Knowledge discovery and data mining
An efficient boosting algorithm for combining preferences
The Journal of Machine Learning Research
Information Retrieval
SVM selective sampling for ranking with application to data retrieval
Proceedings of the eleventh ACM SIGKDD international conference on Knowledge discovery in data mining
Large Margin Methods for Structured and Interdependent Output Variables
The Journal of Machine Learning Research
Learning to rank using gradient descent
ICML '05 Proceedings of the 22nd international conference on Machine learning
Adapting ranking SVM to document retrieval
SIGIR '06 Proceedings of the 29th annual international ACM SIGIR conference on Research and development in information retrieval
Learning to rank: from pairwise approach to listwise approach
Proceedings of the 24th international conference on Machine learning
A support vector method for optimizing average precision
SIGIR '07 Proceedings of the 30th annual international ACM SIGIR conference on Research and development in information retrieval
Ranking with multiple hyperplanes
SIGIR '07 Proceedings of the 30th annual international ACM SIGIR conference on Research and development in information retrieval
FRank: a ranking method with fidelity loss
SIGIR '07 Proceedings of the 30th annual international ACM SIGIR conference on Research and development in information retrieval
AdaRank: a boosting algorithm for information retrieval
SIGIR '07 Proceedings of the 30th annual international ACM SIGIR conference on Research and development in information retrieval
SIGIR '07 Proceedings of the 30th annual international ACM SIGIR conference on Research and development in information retrieval
SoftRank: optimizing non-smooth rank metrics
WSDM '08 Proceedings of the 2008 International Conference on Web Search and Data Mining
Subset ranking using regression
COLT'06 Proceedings of the 19th annual conference on Learning Theory
Ranking with ordered weighted pairwise classification
ICML '09 Proceedings of the 26th Annual International Conference on Machine Learning
Learning to Rank for Information Retrieval
Foundations and Trends in Information Retrieval
Ranking model adaptation for domain-specific search
Proceedings of the 18th ACM conference on Information and knowledge management
Learning to rank from Bayesian decision inference
Proceedings of the 18th ACM conference on Information and knowledge management
A brief survey of computational approaches in social computing
IJCNN'09 Proceedings of the 2009 international joint conference on Neural Networks
LETOR: A benchmark collection for research on learning to rank for information retrieval
Information Retrieval
Cross-market model adaptation with pairwise preference data for web search ranking
COLING '10 Proceedings of the 23rd International Conference on Computational Linguistics: Posters
Learning multiple metrics for ranking
Frontiers of Computer Science in China
Multi-task learning to rank for web search
Pattern Recognition Letters
Leveraging Auxiliary Data for Learning to Rank
ACM Transactions on Intelligent Systems and Technology (TIST)
Proceedings of the fifth ACM international conference on Web search and data mining
A Learning to Rank framework applied to text-image retrieval
Multimedia Tools and Applications
TFMAP: optimizing MAP for top-n context-aware recommendation
SIGIR '12 Proceedings of the 35th international ACM SIGIR conference on Research and development in information retrieval
CLiMF: learning to maximize reciprocal rank with collaborative less-is-more filtering
Proceedings of the sixth ACM conference on Recommender systems
Mining large streams of user data for personalized recommendations
ACM SIGKDD Explorations Newsletter
Learning to name faces: a multimodal learning scheme for search-based face annotation
Proceedings of the 36th international ACM SIGIR conference on Research and development in information retrieval
Direct optimization of ranking measures for learning to rank models
Proceedings of the 19th ACM SIGKDD international conference on Knowledge discovery and data mining
GAPfm: optimal top-n recommendations for graded relevance domains
Proceedings of the 22nd ACM international conference on Conference on information & knowledge management
Proceedings of the 23rd international conference on World wide web
Hi-index | 0.00 |
One of the central issues in learning to rank for information retrieval is to develop algorithms that construct ranking models by directly optimizing evaluation measures used in information retrieval such as Mean Average Precision (MAP) and Normalized Discounted Cumulative Gain (NDCG). Several such algorithms including SVMmap and AdaRank have been proposed and their effectiveness has been verified. However, the relationships between the algorithms are not clear, and furthermore no comparisons have been conducted between them. In this paper, we conduct a study on the approach of directly optimizing evaluation measures in learning to rank for Information Retrieval (IR). We focus on the methods that minimize loss functions upper bounding the basic loss function defined on the IR measures. We first provide a general framework for the study and analyze the existing algorithms of SVMmap and AdaRank within the framework. The framework is based on upper bound analysis and two types of upper bounds are discussed. Moreover, we show that we can derive new algorithms on the basis of this analysis and create one example algorithm called PermuRank. We have also conducted comparisons between SVMmap, AdaRank, PermuRank, and conventional methods of Ranking SVM and RankBoost, using benchmark datasets. Experimental results show that the methods based on direct optimization of evaluation measures can always outperform conventional methods of Ranking SVM and RankBoost. However, no significant difference exists among the performances of the direct optimization methods themselves.