Information visualization: perception for design
Information visualization: perception for design
Crowdsourcing user studies with Mechanical Turk
Proceedings of the SIGCHI Conference on Human Factors in Computing Systems
Cheap and fast---but is it good?: evaluating non-expert annotations for natural language tasks
EMNLP '08 Proceedings of the Conference on Empirical Methods in Natural Language Processing
Crowdsourcing, attention and productivity
Journal of Information Science
Are your participants gaming the system?: screening mechanical turk workers
Proceedings of the SIGCHI Conference on Human Factors in Computing Systems
Financial incentives and the "performance of crowds"
ACM SIGKDD Explorations Newsletter
Soylent: a word processor with a crowd inside
UIST '10 Proceedings of the 23nd annual ACM symposium on User interface software and technology
Designing incentives for inexpert human raters
Proceedings of the ACM 2011 conference on Computer supported cooperative work
CrowdForge: crowdsourcing complex work
CHI '11 Extended Abstracts on Human Factors in Computing Systems
Collaboratively crowdsourcing workflows with turkomatic
Proceedings of the ACM 2012 conference on Computer Supported Cooperative Work
| Hi-index | 0.00 |
Interdependent tasks in Mechanical Turk (MTurk) can be managed efficiently with a workflow, a sequence of tasks through which work passes to its completion. We ask if workers should be informed about the workflow, which we call workflow transparency. Transparency could motivate workers or induce social loafing. We describe three experiments to determine the effects of workflow transparency in MTurk. We compared a text description of the workflow, a visualization of the workflow, and the combination of text and visualization with a control condition giving no workflow information. Workflow transparency marginally increased volunteerism on a charity identification task (experiment 1) and significantly increased volunteerism and quality on a business identification task (experiment 2). Results were weaker with a less experienced worker sample (experiment 3). We suggest further research on the design of workflow information to increase workers' motivation.