Online crowdsourcing subjective image quality assessment

  • Authors:
  • Qianqian Xu;Qingming Huang;Yuan Yao

  • Affiliations:
  • Graduate University, Chinese Academy of Sciences, Beijing, China;Graduate University, Chinese Academy of Sciences, Beijing, China;School of Mathematical Sciences, LMAM and LMP, Peking University, Beijing, China

  • Venue:
  • Proceedings of the 20th ACM international conference on Multimedia
  • Year:
  • 2012

Quantified Score

Hi-index 0.00

Visualization

Abstract

Recently, HodgeRank on random graphs has been proposed as an effective framework for multimedia quality assessment problem based on paired comparison method. With the random design on large graphs, it is particularly suitable for large scale crowdsourcing experiments on Internet. However, to make it more practical toward this purpose, it is necessary to develop online algorithms to deal with sequential or streaming data. In this paper, we propose an online rating scheme based on HodgeRank on random graphs, to assess image quality when assessors and image pairs enter the system in a sequential way in a crowdsourceable scenario. The scheme is shown in both theory and experiments to be effective by exhibiting similar performance to batch learning under the Erdös-Rényi random graph model for sampling. It enables us to derive global rating and monitor intrinsic inconsistency in the real time. We demonstrate the effectiveness of the proposed framework on LIVE and IVC databases.