Labeling images with a computer game
Proceedings of the SIGCHI Conference on Human Factors in Computing Systems
Becoming Wikipedian: transformation of participation in a collaborative online encyclopedia
GROUP '05 Proceedings of the 2005 international ACM SIGGROUP conference on Supporting group work
Improving accessibility of the web with a computer game
Proceedings of the SIGCHI Conference on Human Factors in Computing Systems
Computer
Crowdsourcing user studies with Mechanical Turk
Proceedings of the SIGCHI Conference on Human Factors in Computing Systems
Harnessing the wisdom of crowds in wikipedia: quality through coordination
Proceedings of the 2008 ACM conference on Computer supported cooperative work
TurKit: tools for iterative tasks on mechanical Turk
Proceedings of the ACM SIGKDD Workshop on Human Computation
Financial incentives and the "performance of crowds"
Proceedings of the ACM SIGKDD Workshop on Human Computation
Cheap and fast---but is it good?: evaluating non-expert annotations for natural language tasks
EMNLP '08 Proceedings of the Conference on Empirical Methods in Natural Language Processing
Crowdsourcing graphical perception: using mechanical turk to assess visualization design
Proceedings of the SIGCHI Conference on Human Factors in Computing Systems
TurKit: human computation algorithms on mechanical turk
UIST '10 Proceedings of the 23nd annual ACM symposium on User interface software and technology
XRDS: Crossroads, The ACM Magazine for Students - Comp-YOU-Ter
Knowledge Aggregation in Human Flesh Search
GREENCOM-CPSCOM '10 Proceedings of the 2010 IEEE/ACM Int'l Conference on Green Computing and Communications & Int'l Conference on Cyber, Physical and Social Computing
Cooks or cobblers?: crowd creativity through combination
Proceedings of the SIGCHI Conference on Human Factors in Computing Systems
Human computation: a survey and taxonomy of a growing field
Proceedings of the SIGCHI Conference on Human Factors in Computing Systems
Turkomatic: automatic recursive task and workflow design for mechanical turk
CHI '11 Extended Abstracts on Human Factors in Computing Systems
The crowdsourcing design space
FAC'11 Proceedings of the 6th international conference on Foundations of augmented cognition: directing the future of adaptive systems
Feature selection in crowd creativity
FAC'11 Proceedings of the 6th international conference on Foundations of augmented cognition: directing the future of adaptive systems
Platemate: crowdsourcing nutritional analysis from food photographs
Proceedings of the 24th annual ACM symposium on User interface software and technology
Crowds in two seconds: enabling realtime crowd-powered interfaces
Proceedings of the 24th annual ACM symposium on User interface software and technology
Minority voices of crowdsourcing: why we should pay attention to every member of the crowd
Proceedings of the ACM 2012 conference on Computer Supported Cooperative Work Companion
Shepherding the crowd yields better work
Proceedings of the ACM 2012 conference on Computer Supported Cooperative Work
Direct answers for search queries in the long tail
Proceedings of the SIGCHI Conference on Human Factors in Computing Systems
Distributed sensemaking: improving sensemaking by leveraging the efforts of previous users
Proceedings of the SIGCHI Conference on Human Factors in Computing Systems
Deploying monotrans widgets in the wild
Proceedings of the SIGCHI Conference on Human Factors in Computing Systems
Medusa: a programming framework for crowd-sensing applications
Proceedings of the 10th international conference on Mobile systems, applications, and services
Tagging tagged images: on the impact of existing annotations on image tagging
Proceedings of the ACM multimedia 2012 workshop on Crowdsourcing for multimedia
CrowdLang: a programming language for the systematic exploration of human computation systems
SocInfo'12 Proceedings of the 4th international conference on Social Informatics
Leveraging transitive relations for crowdsourced joins
Proceedings of the 2013 ACM SIGMOD International Conference on Management of Data
Proceedings of the 5th ACM SIGCHI symposium on Engineering interactive computing systems
TaskGenies: Automatically Providing Action Plans Helps People Complete Tasks
ACM Transactions on Computer-Human Interaction (TOCHI)
AskSheet: efficient human computation for decision making with spreadsheets
Proceedings of the 17th ACM conference on Computer supported cooperative work & social computing
Crowdsourcing traffic simulation to improve signal timing
Journal of Computing Sciences in Colleges
Hi-index | 0.00 |
Services like Amazon's Mechanical Turk have opened the door for exploration of processes that outsource computation to humans. These human computation processes hold tremendous potential to solve a variety of problems in novel and interesting ways. However, we are only just beginning to understand how to design such processes. This paper explores two basic approaches: one where workers work alone in parallel and one where workers iteratively build on each other's work. We present a series of experiments exploring tradeoffs between each approach in several problem domains: writing, brainstorming, and transcription. In each of our experiments, iteration increases the average quality of responses. The increase is statistically significant in writing and brainstorming. However, in brainstorming and transcription, it is not clear that iteration is the best overall approach, in part because both of these tasks benefit from a high variability of responses, which is more prevalent in the parallel process. Also, poor guesses in the transcription task can lead subsequent workers astray.