Labeling images with a computer game
Proceedings of the SIGCHI Conference on Human Factors in Computing Systems
Why we tag: motivations for annotation in mobile and online media
Proceedings of the SIGCHI Conference on Human Factors in Computing Systems
Crowdsourcing and the question of expertise
Communications of the ACM - Finding the Fun in Computer Science Education
Proceedings of the international conference on Multimedia information retrieval
Automatic image semantic interpretation using social action and tagging data
Multimedia Tools and Applications
Modeling Human Judgment of Digital Imagery for Multimedia Retrieval
IEEE Transactions on Multimedia
Efficient annotation of image data sets for computer vision applications
Proceedings of the 1st International Workshop on Visual Interfaces for Ground Truth Collection in Computer Vision Applications
Ground truth generation in medical imaging: a crowdsourcing-based iterative approach
Proceedings of the ACM multimedia 2012 workshop on Crowdsourcing for multimedia
Competitive affective gaming: winning with a smile
Proceedings of the 21st ACM international conference on Multimedia
| Hi-index | 0.00 |
Affective-interaction in computer games is a novel area with several new challenges, such as detecting players' facial expressions (e.g., happy, sad, surprise) in a robust manner. In this paper we describe a crowdsourcing effort for creating the ground-truth of a large-scale dataset of images capturing users playing a computer game. The computer game is designed to elicit a particular facial expressions and the game will score the player according to the detected expression. For designing the crowdsourcing task, some of the examined variables include: reward, tagging limits, golden questions, workers' location. In the end, we designed a large tagging job to maximize workers agreement. Each image with a facial expressions is tagged with one of the following expressions labels: happy, anger, disgust, contempt, sad, fear, surprise, and neutral. The dataset included over 40,000 images, the workers' judgments, the game's detected facial expression and what facial expression the player should be performing.