ROC analysis in ordinal regression learning
Pattern Recognition Letters
On the scalability of ordered multi-class ROC analysis
Computational Statistics & Data Analysis
The Journal of Machine Learning Research
Hinge Rank Loss and the Area Under the ROC Curve
ECML '07 Proceedings of the 18th European conference on Machine Learning
Approximation of the Optimal ROC Curve and a Tree-Based Ranking Algorithm
ALT '08 Proceedings of the 19th international conference on Algorithmic Learning Theory
The ROC isometrics approach to construct reliable classifiers
Intelligent Data Analysis
An efficient projection for l1, ∞ regularization
ICML '09 Proceedings of the 26th Annual International Conference on Machine Learning
Learning to Rank for Information Retrieval
Foundations and Trends in Information Retrieval
IEEE Transactions on Information Theory
MINLIP: Efficient Learning of Transformation Models
ICANN '09 Proceedings of the 19th International Conference on Artificial Neural Networks: Part I
Margin-based Ranking and an Equivalence between AdaBoost and RankBoost
The Journal of Machine Learning Research
The P-Norm Push: A Simple Convex Ranking Algorithm that Concentrates at the Top of the List
The Journal of Machine Learning Research
Nested support vector machines
IEEE Transactions on Signal Processing
Adaptive estimation of the optimal ROC curve and a bipartite ranking algorithm
ALT'09 Proceedings of the 20th international conference on Algorithmic learning theory
A transitivity analysis of bipartite rankings in pairwise multi-class classification
Information Sciences: an International Journal
Supervised multidimensional scaling for visualization, classification, and bipartite ranking
Computational Statistics & Data Analysis
The Journal of Machine Learning Research
Computational Statistics & Data Analysis
On the ERA ranking representability of pairwise bipartite ranking functions
Artificial Intelligence
Multiview semi-supervised learning for ranking multilingual documents
ECML PKDD'11 Proceedings of the 2011 European conference on Machine learning and knowledge discovery in databases - Volume Part III
COLT'06 Proceedings of the 19th annual conference on Learning Theory
Subset ranking using regression
COLT'06 Proceedings of the 19th annual conference on Learning Theory
Learnability of bipartite ranking functions
COLT'05 Proceedings of the 18th annual conference on Learning Theory
Stability and generalization of bipartite ranking algorithms
COLT'05 Proceedings of the 18th annual conference on Learning Theory
Margin-Based ranking meets boosting in the middle
COLT'05 Proceedings of the 18th annual conference on Learning Theory
Efficient rank aggregation using partial data
Proceedings of the 12th ACM SIGMETRICS/PERFORMANCE joint international conference on Measurement and Modeling of Computer Systems
On ranking and generalization bounds
The Journal of Machine Learning Research
Full length article: The convergence rate of a regularized ranking algorithm
Journal of Approximation Theory
Information Sciences: an International Journal
Efficient multifaceted screening of job applicants
Proceedings of the 16th International Conference on Extending Database Technology
Effect on generalization of using relational information in list-wise algorithms
ICPCA/SWS'12 Proceedings of the 2012 international conference on Pervasive Computing and the Networked World
Proceedings of the 19th ACM SIGKDD international conference on Knowledge discovery and data mining
The Journal of Machine Learning Research
Uniform convergence, stability and learnability for ranking problems
IJCAI'13 Proceedings of the Twenty-Third international joint conference on Artificial Intelligence
Hi-index | 0.06 |
We study generalization properties of the area under the ROC curve (AUC), a quantity that has been advocated as an evaluation criterion for the bipartite ranking problem.The AUC is a different term than the error rate used for evaluation in classification problems; consequently, existing generalization bounds for the classification error rate cannot be used to draw conclusions about the AUC.In this paper, we define the expected accuracy of a ranking function (analogous to the expected error rate of a classification function), and derive distribution-free probabilistic bounds on the deviation of the empirical AUC of a ranking function (observed on a finite data sequence) from its expected accuracy.We derive both a large deviation bound, which serves to bound the expected accuracy of a ranking function in terms of its empirical AUC on a test sequence, and a uniform convergence bound, which serves to bound the expected accuracy of a learned ranking function in terms of its empirical AUC on a training sequence.Our uniform convergence bound is expressed in terms of a new set of combinatorial parameters that we term the bipartite rank-shatter coefficients; these play the same role in our result as do the standard VC-dimension related shatter coefficients (also known as the growth function) in uniform convergence results for the classification error rate. A comparison of our result with a recent uniform convergence result derived by Freund et al. (2003) for a quantity closely related to the AUC shows that the bound provided by our result can be considerably tighter.