Distortion as a validation criterion in the identification of suspicious reviews

  • Authors:
  • Guangyu Wu;Derek Greene;Barry Smyth;Pádraig Cunningham

  • Affiliations:
  • University College Dublin, Ireland;University College Dublin, Ireland;University College Dublin, Ireland;University College Dublin, Ireland

  • Venue:
  • Proceedings of the First Workshop on Social Media Analytics
  • Year:
  • 2010

Quantified Score

Hi-index 0.00

Visualization

Abstract

Assessing the trustworthiness of reviews is a key issue for the maintainers of opinion sites such as TripAdvisor. In this paper we propose a distortion criterion for assessing the impact of methods for uncovering suspicious hotel reviews in TripAdvisor. The principle is that dishonest reviews will distort the overall popularity ranking for a collection of hotels. Thus a mechanism that deletes dishonest reviews will distort the popularity ranking significantly, when compared with the removal of a similar set of reviews at random. This distortion can be quantified by comparing popularity rankings before and after deletion, using rank correlation. We present an evaluation of this strategy in the assessment of shill detection mechanisms on a dataset of hotel reviews collected from TripAdvisor.