ROC curves for regression

Authors:
José Hernández-Orallo
Affiliations:
-
Venue:
Pattern Recognition
Year:
2013

Citing 25
Cited 0

Learning Decision Trees Using the Area Under the ROC Curve

ICML '02 Proceedings of the Nineteenth International Conference on Machine Learning
An empirical study of maintenance and development estimation accuracy

Journal of Systems and Software
Regression error characteristic surfaces

Proceedings of the eleventh ACM SIGKDD international conference on Knowledge discovery in data mining
The use of receiver operating characteristic curves in biomedical informatics

Journal of Biomedical Informatics - Special issue: Clinical machine learning
An introduction to ROC analysis

Pattern Recognition Letters - Special issue: ROC analysis in pattern recognition
Cost curves: An improved method for visualizing classifier performance

Machine Learning
Selecting features in microarray classification using ROC curves

Pattern Recognition
Ranking-based evaluation of regression models

Knowledge and Information Systems
Maximizing the area under the ROC curve by pairwise feature combination

Pattern Recognition
Maximizing area under ROC curve for biometric scores fusion

Pattern Recognition
An experimental comparison of performance measures for classification

Pattern Recognition Letters
Tuning Data Mining Methods for Cost-Sensitive Regression: A Study in Loan Charge-Off Forecasting

Journal of Management Information Systems
ROC Curves for Continuous Data

ROC Curves for Continuous Data
Measuring classifier performance: a coherent alternative to the area under the ROC curve

Machine Learning
The foundations of cost-sensitive learning

IJCAI'01 Proceedings of the 17th international joint conference on Artificial intelligence - Volume 2
An extension of the Gauss-Newton algorithm for estimation under asymmetric loss

Computational Statistics & Data Analysis
The use of the area under the ROC curve in the evaluation of machine learning algorithms

Pattern Recognition
Iterative Boolean combination of classifiers in the ROC space: An application to anomaly detection with HMMs

Pattern Recognition
Machine Learning Methods and Asymmetric Cost Function to Estimate Execution Effort of Software Testing

ICST '10 Proceedings of the 2010 Third International Conference on Software Testing, Verification and Validation
The ROC manifold for classification systems

Pattern Recognition
An extended tuning method for cost-sensitive regression and forecasting

Decision Support Systems
Partial AUC maximization in a linear combination of dichotomizers

Pattern Recognition
Adaptive ROC-based ensembles of HMMs applied to anomaly detection

Pattern Recognition
An online AUC formulation for binary classification

Pattern Recognition
A unified view of performance metrics: translating threshold choice into expected classification loss

The Journal of Machine Learning Research

Quantified Score

Hi-index	0.01

Visualization

Abstract

Receiver Operating Characteristic (ROC) analysis is one of the most popular tools for the visual assessment and understanding of classifier performance. In this paper we present a new representation of regression models in the so-called regression ROC (RROC) space. The basic idea is to represent over-estimation against under-estimation. The curves are just drawn by adjusting a shift, a constant that is added (or subtracted) to the predictions, and plays a similar role as a threshold in classification. From here, we develop the notions of optimal operating condition, convexity, dominance, and explore several evaluation metrics that can be shown graphically, such as the area over the RROC curve (AOC). In particular, we show a novel and significant result: the AOC is equivalent to the error variance. We illustrate the application of RROC curves to resource estimation, namely the estimation of software project effort.