Prediction of favourite photos using social, visual, and textual signals

  • Authors:
  • Roelof van Zwol;Adam Rae;Lluis Garcia Pueyo

  • Affiliations:
  • Yahoo! Research, Santa Clara, CA, USA;Open University, Milton Keynes, United Kingdom;Yahoo! Research, Barcelona, Spain

  • Venue:
  • Proceedings of the international conference on Multimedia
  • Year:
  • 2010

Quantified Score

Hi-index 0.00

Visualization

Abstract

This paper focuses on the prediction of users' favourite photos in Flickr. We propose a multi-modal, machine learned approach that combines social, visual and textual signals into a single prediction system. Although each individual user has different motivations for calling a photo a favourite, we show that the textual, visual, and social modalities effectively capture the needs of most active Flickr users. We use gradient-boosted decision trees (GBDT) with a mod least squares loss function for the classification of a user's favourite photos, and evaluate the performance of our classifier with respect to the individual modalities and various combinations thereof. By using a combination of the social and visual modalities the GBDT creates a highly effective classifier. The addition of textual features allows us to significantly increase recall, with a slight trade off in precision.