Regression conformal prediction with nearest neighbours

  • Authors:
  • Harris Papadopoulos;Vladimir Vovk;Alex Gammerman

  • Affiliations:
  • Computer Science and Engineering Department, Frederick University, Palouriotisa, Nicosia, Cyprus;Computer Learning Research Centre, Department of Computer Science, University of London, Egham, Surrey, UK;Computer Learning Research Centre, Department of Computer Science, University of London, Egham, Surrey, UK

  • Venue:
  • Journal of Artificial Intelligence Research
  • Year:
  • 2011

Quantified Score

Hi-index 0.00

Visualization

Abstract

In this paper we apply Conformal Prediction (CP) to the k-Nearest Neighbours Regression (k-NNR) algorithm and propose ways of extending the typical nonconformity measure used for regression so far. Unlike traditional regression methods which produce point predictions, Conformal Predictors output predictive regions that satisfy a given confidence level. The regions produced by any Conformal Predictor are automatically valid, however their tightness and therefore usefulness depends on the nonconformity measure used by each CP. In effect a nonconformity measure evaluates how strange a given example is compared to a set of other examples based on some traditional machine learning algorithm. We define six novel nonconformity measures based on the k-Nearest Neighbours Regression algorithm and develop the corresponding CPs following both the original (transductive) and the inductive CP approaches. A comparison of the predictive regions produced by our measures with those of the typical regression measure suggests that a major improvement in terms of predictive region tightness is achieved by the new measures.