Collaborative quality filtering: establishing consensus or recovering ground truth?

  • Authors:
  • Jonathan Traupman;Robert Wilensky

  • Affiliations:
  • Computer Science Division, University of California, Berkeley, CA;Computer Science Division, University of California, Berkeley, CA

  • Venue:
  • WebKDD'04 Proceedings of the 6th international conference on Knowledge Discovery on the Web: advances in Web Mining and Web Usage Analysis
  • Year:
  • 2004

Quantified Score

Hi-index 0.00

Visualization

Abstract

We present a algorithm based on factor analysis for performing collaborative quality filtering (CQF). Unlike previous approaches to CQF, which estimate the consensus opinion of a group of reviewers, our algorithm uses a generative model of the review process to estimate the latent intrinsic quality of the items under reviews. We run several tests that demonstrate that consensus and intrinsic quality are, in fact different and unrelated aspects of quality. These results suggest that asymptotic consensus, which purports to model peer review, is, in fact, not recovering the ground truth quality of reviewed items.