IR evaluation methods for retrieving highly relevant documents
SIGIR '00 Proceedings of the 23rd annual international ACM SIGIR conference on Research and development in information retrieval
Neural Networks: Tricks of the Trade, this book is an outgrowth of a 1996 NIPS workshop
Optimizing search engines using clickthrough data
Proceedings of the eighth ACM SIGKDD international conference on Knowledge discovery and data mining
Simple BM25 extension to multiple weighted fields
Proceedings of the thirteenth ACM international conference on Information and knowledge management
Gaussian Processes for Ordinal Regression
The Journal of Machine Learning Research
Learning to rank using gradient descent
ICML '05 Proceedings of the 22nd international conference on Machine learning
Improving web search ranking by incorporating user behavior information
SIGIR '06 Proceedings of the 29th annual international ACM SIGIR conference on Research and development in information retrieval
Optimisation methods for ranking functions with multiple parameters
CIKM '06 Proceedings of the 15th ACM international conference on Information and knowledge management
Learning to rank: from pairwise approach to listwise approach
Proceedings of the 24th international conference on Machine learning
Learning diverse rankings with multi-armed bandits
Proceedings of the 25th international conference on Machine learning
Directly optimizing evaluation measures in learning to rank
Proceedings of the 31st annual international ACM SIGIR conference on Research and development in information retrieval
Learning to rank with SoftRank and Gaussian processes
Proceedings of the 31st annual international ACM SIGIR conference on Research and development in information retrieval
Structured learning for non-smooth ranking losses
Proceedings of the 14th ACM SIGKDD international conference on Knowledge discovery and data mining
Estimating retrieval effectiveness using rank distributions
Proceedings of the 17th ACM conference on Information and knowledge management
Proceedings of the Second ACM International Conference on Web Search and Data Mining
Ranking with ordered weighted pairwise classification
ICML '09 Proceedings of the 26th Annual International Conference on Machine Learning
BoltzRank: learning to maximize expected ranking gain
ICML '09 Proceedings of the 26th Annual International Conference on Machine Learning
Robust sparse rank learning for non-smooth ranking measures
Proceedings of the 32nd international ACM SIGIR conference on Research and development in information retrieval
Comparing both relevance and robustness in selection of web ranking functions
Proceedings of the 32nd international ACM SIGIR conference on Research and development in information retrieval
Learning to Rank for Information Retrieval
Foundations and Trends in Information Retrieval
Smoothing DCG for learning to rank: a novel approach using smoothed hinge functions
Proceedings of the 18th ACM conference on Information and knowledge management
Incorporating robustness into web ranking evaluation
Proceedings of the 18th ACM conference on Information and knowledge management
IntervalRank: isotonic regression with listwise and pairwise constraints
Proceedings of the third ACM international conference on Web search and data mining
Book search experiments: investigating IR methods for the indexing and retrieval of books
ECIR'08 Proceedings of the IR research, 30th European conference on Advances in information retrieval
On the choice of effectiveness measures for learning to rank
Information Retrieval
Gradient descent optimization of smoothed information retrieval metrics
Information Retrieval
Introduction to special issue on learning to rank for information retrieval
Information Retrieval
On statistical analysis and optimization of information retrieval effectiveness metrics
Proceedings of the 33rd international ACM SIGIR conference on Research and development in information retrieval
Extending average precision to graded relevance judgments
Proceedings of the 33rd international ACM SIGIR conference on Research and development in information retrieval
LETOR: A benchmark collection for research on learning to rank for information retrieval
Information Retrieval
Directly optimizing evaluation measures in learning to rank based on the clonal selection algorithm
CIKM '10 Proceedings of the 19th ACM international conference on Information and knowledge management
Ranking continuous probabilistic datasets
Proceedings of the VLDB Endowment
Learning to re-rank web search results with multiple pairwise features
Proceedings of the fourth ACM international conference on Web search and data mining
Learning to rank with document ranks and scores
Knowledge-Based Systems
Learning to rank with multiple objective functions
Proceedings of the 20th international conference on World wide web
A stochastic learning-to-rank algorithm and its application to contextual advertising
Proceedings of the 20th international conference on World wide web
Parallel boosted regression trees for web search ranking
Proceedings of the 20th international conference on World wide web
Transductive learning to rank using association rules
Expert Systems with Applications: An International Journal
Learning multiple metrics for ranking
Frontiers of Computer Science in China
Learning to rank with nonlinear monotonic ensemble
MCS'11 Proceedings of the 10th international conference on Multiple classifier systems
Smoothing NDCG metrics using tied scores
Proceedings of the 20th ACM international conference on Information and knowledge management
Maximum margin ranking algorithms for information retrieval
ECIR'2010 Proceedings of the 32nd European conference on Advances in Information Retrieval
A Learning to Rank framework applied to text-image retrieval
Multimedia Tools and Applications
TFMAP: optimizing MAP for top-n context-aware recommendation
SIGIR '12 Proceedings of the 35th international ACM SIGIR conference on Research and development in information retrieval
Extending BM25 with multiple query operators
SIGIR '12 Proceedings of the 35th international ACM SIGIR conference on Research and development in information retrieval
Multiple objective optimization in recommender systems
Proceedings of the sixth ACM conference on Recommender systems
CLiMF: learning to maximize reciprocal rank with collaborative less-is-more filtering
Proceedings of the sixth ACM conference on Recommender systems
Variance maximization via noise injection for active sampling in learning to rank
Proceedings of the 21st ACM international conference on Information and knowledge management
Direct optimization of ranking measures for learning to rank models
Proceedings of the 19th ACM SIGKDD international conference on Knowledge discovery and data mining
Efficient gradient descent algorithm for sparse models with application in learning-to-rank
Knowledge-Based Systems
CTR prediction for contextual advertising: learning-to-rank approach
Proceedings of the Seventh International Workshop on Data Mining for Online Advertising
Ranked bandits in metric spaces: learning diverse rankings over large document collections
The Journal of Machine Learning Research
GAPfm: optimal top-n recommendations for graded relevance domains
Proceedings of the 22nd ACM international conference on Conference on information & knowledge management
Democracy is good for ranking: towards multi-view rank learning and adaptation in web search
Proceedings of the 7th ACM international conference on Web search and data mining
Proceedings of the 23rd international conference on World wide web
| Hi-index | 0.00 |
We address the problem of learning large complex ranking functions. Most IR applications use evaluation metrics that depend only upon the ranks of documents. However, most ranking functions generate document scores, which are sorted to produce a ranking. Hence IR metrics are innately non-smooth with respect to the scores, due to the sort. Unfortunately, many machine learning algorithms require the gradient of a training objective in order to perform the optimization of the model parameters,and because IR metrics are non-smooth,we need to find a smooth proxy objective that can be used for training. We present a new family of training objectives that are derived from the rank distributions of documents, induced by smoothed scores. We call this approach SoftRank. We focus on a smoothed approximation to Normalized Discounted Cumulative Gain (NDCG), called SoftNDCG and we compare it with three other training objectives in the recent literature. We present two main results. First, SoftRank yields a very good way of optimizing NDCG. Second, we show that it is possible to achieve state of the art test set NDCG results by optimizing a soft NDCG objective on the training set with a different discount function