Semi-parametric analysis of multi-rater data

  • Authors:
  • Simon Rogers;Mark Girolami;Tamara Polajnar

  • Affiliations:
  • Department of Computing Science, University of Glasgow, Glasgow, UK G12 8QQ;Department of Computing Science, University of Glasgow, Glasgow, UK G12 8QQ;Department of Computing Science, University of Glasgow, Glasgow, UK G12 8QQ

  • Venue:
  • Statistics and Computing
  • Year:
  • 2010

Quantified Score

Hi-index 0.00

Visualization

Abstract

Datasets that are subjectively labeled by a number of experts are becoming more common in tasks such as biological text annotation where class definitions are necessarily somewhat subjective. Standard classification and regression models are not suited to multiple labels and typically a pre-processing step (normally assigning the majority class) is performed. We propose Bayesian models for classification and ordinal regression that naturally incorporate multiple expert opinions in defining predictive distributions. The models make use of Gaussian process priors, resulting in great flexibility and particular suitability to text based problems where the number of covariates can be far greater than the number of data instances. We show that using all labels rather than just the majority improves performance on a recent biological dataset.