OHSUMED: an interactive retrieval evaluation and new large test collection for research
SIGIR '94 Proceedings of the 17th annual international ACM SIGIR conference on Research and development in information retrieval
IR evaluation methods for retrieving highly relevant documents
SIGIR '00 Proceedings of the 23rd annual international ACM SIGIR conference on Research and development in information retrieval
Modern Information Retrieval
Liberal relevance criteria of TREC -: counting on negligible documents?
SIGIR '02 Proceedings of the 25th annual international ACM SIGIR conference on Research and development in information retrieval
Cumulated gain-based evaluation of IR techniques
ACM Transactions on Information Systems (TOIS)
Optimizing search engines using clickthrough data
Proceedings of the eighth ACM SIGKDD international conference on Knowledge discovery and data mining
The concept of relevance in IR
Journal of the American Society for Information Science and Technology
An efficient boosting algorithm for combining preferences
The Journal of Machine Learning Research
Discriminative models for information retrieval
Proceedings of the 27th annual international ACM SIGIR conference on Research and development in information retrieval
Exploiting the hierarchical structure for link analysis
Proceedings of the 28th annual international ACM SIGIR conference on Research and development in information retrieval
A study of relevance propagation for web search
Proceedings of the 28th annual international ACM SIGIR conference on Research and development in information retrieval
Learning to rank using gradient descent
ICML '05 Proceedings of the 22nd international conference on Machine learning
Adapting ranking SVM to document retrieval
SIGIR '06 Proceedings of the 29th annual international ACM SIGIR conference on Research and development in information retrieval
High accuracy retrieval with multiple nested ranker
SIGIR '06 Proceedings of the 29th annual international ACM SIGIR conference on Research and development in information retrieval
Learning to rank: from pairwise approach to listwise approach
Proceedings of the 24th international conference on Machine learning
Ranking with multiple hyperplanes
SIGIR '07 Proceedings of the 30th annual international ACM SIGIR conference on Research and development in information retrieval
FRank: a ranking method with fidelity loss
SIGIR '07 Proceedings of the 30th annual international ACM SIGIR conference on Research and development in information retrieval
Learning to rank relational objects and its application to web search
Proceedings of the 17th international conference on World Wide Web
Query-level stability and generalization in learning to rank
Proceedings of the 25th international conference on Machine learning
Listwise approach to learning to rank: theory and algorithm
Proceedings of the 25th international conference on Machine learning
Generalization analysis of listwise learning-to-rank algorithms
ICML '09 Proceedings of the 26th Annual International Conference on Machine Learning
On the local optimality of LambdaRank
Proceedings of the 32nd international ACM SIGIR conference on Research and development in information retrieval
Learning to Rank for Information Retrieval
Foundations and Trends in Information Retrieval
Learning to rank from Bayesian decision inference
Proceedings of the 18th ACM conference on Information and knowledge management
LETOR: A benchmark collection for research on learning to rank for information retrieval
Information Retrieval
RankDE: learning a ranking function for information retrieval using differential evolution
Proceedings of the 13th annual conference on Genetic and evolutionary computation
Learning to rank using query-level regression
Proceedings of the 34th international ACM SIGIR conference on Research and development in Information Retrieval
ListOPT: learning to optimize for XML ranking
PAKDD'11 Proceedings of the 15th Pacific-Asia conference on Advances in knowledge discovery and data mining - Volume Part II
Maximum margin ranking algorithms for information retrieval
ECIR'2010 Proceedings of the 32nd European conference on Advances in Information Retrieval
Collaborative ranking: a case study on entity linking
EMNLP '11 Proceedings of the Conference on Empirical Methods in Natural Language Processing
Learning to rank documents using similarity information between objects
ICONIP'11 Proceedings of the 18th international conference on Neural Information Processing - Volume Part II
Cost-Sensitive listwise ranking approach
PAKDD'10 Proceedings of the 14th Pacific-Asia conference on Advances in Knowledge Discovery and Data Mining - Volume Part I
Proceedings of the 2012 Joint EDBT/ICDT Workshops
Effect on generalization of using relational information in list-wise algorithms
ICPCA/SWS'12 Proceedings of the 2012 international conference on Pervasive Computing and the Networked World
Direct optimization of ranking measures for learning to rank models
Proceedings of the 19th ACM SIGKDD international conference on Knowledge discovery and data mining
Whom to mention: expand the diffusion of tweets by @ recommendation on micro-blogging systems
Proceedings of the 22nd international conference on World Wide Web
Is top-k sufficient for ranking?
Proceedings of the 22nd ACM international conference on Conference on information & knowledge management
Democracy is good for ranking: towards multi-view rank learning and adaptation in web search
Proceedings of the 7th ACM international conference on Web search and data mining
Hi-index | 0.00 |
Many machine learning technologies such as support vector machines, boosting, and neural networks have been applied to the ranking problem in information retrieval. However, since originally the methods were not developed for this task, their loss functions do not directly link to the criteria used in the evaluation of ranking. Specifically, the loss functions are defined on the level of documents or document pairs, in contrast to the fact that the evaluation criteria are defined on the level of queries. Therefore, minimizing the loss functions does not necessarily imply enhancing ranking performances. To solve this problem, we propose using query-level loss functions in learning of ranking functions. We discuss the basic properties that a query-level loss function should have and propose a query-level loss function based on the cosine similarity between a ranking list and the corresponding ground truth. We further design a coordinate descent algorithm, referred to as RankCosine, which utilizes the proposed loss function to create a generalized additive ranking model. We also discuss whether the loss functions of existing ranking algorithms can be extended to query-level. Experimental results on the datasets of TREC web track, OHSUMED, and a commercial web search engine show that with the use of the proposed query-level loss function we can significantly improve ranking accuracies. Furthermore, we found that it is difficult to extend the document-level loss functions to query-level loss functions.