Labeling images with a computer game
Proceedings of the SIGCHI Conference on Human Factors in Computing Systems
Motivations of contributors to Wikipedia
ACM SIGCAS Computers and Society
Keyword Generation for Search Engine Advertising
ICDMW '06 Proceedings of the Sixth IEEE International Conference on Data Mining - Workshops
Keyword generation for search engine advertising using semantic similarity between terms
Proceedings of the ninth international conference on Electronic commerce
Communications of the ACM
Using the wisdom of the crowds for keyword generation
Proceedings of the 17th international conference on World Wide Web
Amplifying community content creation with mixed initiative information extraction
Proceedings of the SIGCHI Conference on Human Factors in Computing Systems
CHI '09 Extended Abstracts on Human Factors in Computing Systems
Motivations of Wikipedia content contributors
Computers in Human Behavior
The Journal of Machine Learning Research
Value of learning in sponsored search auctions
WINE'10 Proceedings of the 6th international conference on Internet and network economics
WINE'10 Proceedings of the 6th international conference on Internet and network economics
On saliency, affect and focused attention
Proceedings of the SIGCHI Conference on Human Factors in Computing Systems
Combining human and machine intelligence in large-scale crowdsourcing
Proceedings of the 11th International Conference on Autonomous Agents and Multiagent Systems - Volume 1
UMAP'12 Proceedings of the 20th international conference on User Modeling, Adaptation, and Personalization
Machine Learning: A Probabilistic Perspective
Machine Learning: A Probabilistic Perspective
User engagement: the network effect matters!
Proceedings of the 21st ACM international conference on Information and knowledge management
Absence time and user engagement: evaluating ranking functions
Proceedings of the sixth ACM international conference on Web search and data mining
Pay by the bit: an information-theoretic metric for collective human judgment
Proceedings of the 2013 conference on Computer supported cooperative work
Incentives, gamification, and game theory: an economic approach to badge design
Proceedings of the fourteenth ACM conference on Electronic commerce
Steering user behavior with badges
Proceedings of the 22nd international conference on World Wide Web
Hi-index | 0.00 |
We describe Quizz, a gamified crowdsourcing system that simultaneously assesses the knowledge of users and acquires new knowledge from them. Quizz operates by asking users to complete short quizzes on specific topics; as a user answers the quiz questions, Quizz estimates the user's competence. To acquire new knowledge, Quizz also incorporates questions for which we do not have a known answer; the answers given by competent users provide useful signals for selecting the correct answers for these questions. Quizz actively tries to identify knowledgeable users on the Internet by running advertising campaigns, effectively leveraging the targeting capabilities of existing, publicly available, ad placement services. Quizz quantifies the contributions of the users using information theory and sends feedback to the advertisingsystem about each user. The feedback allows the ad targeting mechanism to further optimize ad placement. Our experiments, which involve over ten thousand users, confirm that we can crowdsource knowledge curation for niche and specialized topics, as the advertising network can automatically identify users with the desired expertise and interest in the given topic. We present controlled experiments that examine the effect of various incentive mechanisms, highlighting the need for having short-term rewards as goals, which incentivize the users to contribute. Finally, our cost-quality analysis indicates that the cost of our approach is below that of hiring workers through paid-crowdsourcing platforms, while offering the additional advantage of giving access to billions of potential users all over the planet, and being able to reach users with specialized expertise that is not typically available through existing labor marketplaces.