Making good probability estimates for regression

  • Authors:
  • Michael Carney;Pádraig Cunningham

  • Affiliations:
  • Trinity College Dublin, Dublin 2, Ireland;Trinity College Dublin, Dublin 2, Ireland

  • Venue:
  • ECML'06 Proceedings of the 17th European conference on Machine Learning
  • Year:
  • 2006

Quantified Score

Hi-index 0.00

Visualization

Abstract

In this paper, we show that the optimisation of density forecasting models for regression in machine learning can be formulated as a multi-objective problem. We describe the two objectives of sharpness and calibration and suggest suitable scoring metrics for both. We use the popular negative log-likelihood as a measure of sharpness and the probability integral transform as a measure of calibration.To optimise density forecasting models under multiple criteria we introduce a multi-objective evolutionary optimisation framework that can produce better density forecasts from a prediction user's perspective. Our experiments show improvements over the state-of-the-art on a risk management problem.