Computer
The Wealth of Networks: How Social Production Transforms Markets and Freedom
The Wealth of Networks: How Social Production Transforms Markets and Freedom
Crowdsourcing user studies with Mechanical Turk
Proceedings of the SIGCHI Conference on Human Factors in Computing Systems
What drives content tagging: the case of photos on Flickr
Proceedings of the SIGCHI Conference on Human Factors in Computing Systems
Crowdsourcing: Why the Power of the Crowd Is Driving the Future of Business
Crowdsourcing: Why the Power of the Crowd Is Driving the Future of Business
Crowdsourcing graphical perception: using mechanical turk to assess visualization design
Proceedings of the SIGCHI Conference on Human Factors in Computing Systems
Re-examining price as a predictor of answer quality in an online q&a site
Proceedings of the SIGCHI Conference on Human Factors in Computing Systems
Enhancing directed content sharing on the web
Proceedings of the SIGCHI Conference on Human Factors in Computing Systems
Presenting diverse political opinions: how and how much
Proceedings of the SIGCHI Conference on Human Factors in Computing Systems
The labor economics of paid crowdsourcing
Proceedings of the 11th ACM conference on Electronic commerce
Task search in a human computation market
Proceedings of the ACM SIGKDD Workshop on Human Computation
Sellers' problems in human computation markets
Proceedings of the ACM SIGKDD Workshop on Human Computation
Exploring iterative and parallel human computation processes
Proceedings of the ACM SIGKDD Workshop on Human Computation
Toward automatic task design: a progress report
Proceedings of the ACM SIGKDD Workshop on Human Computation
Exploring the use of crowdsourcing to support empirical studies in software engineering
Proceedings of the 2010 ACM-IEEE International Symposium on Empirical Software Engineering and Measurement
Examining micro-payments for participatory sensing data collections
Proceedings of the 12th ACM international conference on Ubiquitous computing
TurKit: human computation algorithms on mechanical turk
UIST '10 Proceedings of the 23nd annual ACM symposium on User interface software and technology
Soylent: a word processor with a crowd inside
UIST '10 Proceedings of the 23nd annual ACM symposium on User interface software and technology
Creating speech and language data with Amazon's Mechanical Turk
CSLDAMT '10 Proceedings of the NAACL HLT 2010 Workshop on Creating Speech and Language Data with Amazon's Mechanical Turk
Consensus versus expertise: a case study of word alignment with Mechanical Turk
CSLDAMT '10 Proceedings of the NAACL HLT 2010 Workshop on Creating Speech and Language Data with Amazon's Mechanical Turk
Opinion mining of Spanish customer comments with non-expert annotations on Mechanical Turk
CSLDAMT '10 Proceedings of the NAACL HLT 2010 Workshop on Creating Speech and Language Data with Amazon's Mechanical Turk
Evaluation of commonsense knowledge with Mechanical Turk
CSLDAMT '10 Proceedings of the NAACL HLT 2010 Workshop on Creating Speech and Language Data with Amazon's Mechanical Turk
Crowdsourcing document relevance assessment with Mechanical Turk
CSLDAMT '10 Proceedings of the NAACL HLT 2010 Workshop on Creating Speech and Language Data with Amazon's Mechanical Turk
Location-based crowdsourcing: extending crowdsourcing to the real world
Proceedings of the 6th Nordic Conference on Human-Computer Interaction: Extending Boundaries
Crowdsourcing the indexing of film and television media
Proceedings of the 73rd ASIS&T Annual Meeting on Navigating Streams in an Information Ecosystem - Volume 47
Reflecting on the DARPA Red Balloon Challenge
Communications of the ACM
Everyone's an influencer: quantifying influence on twitter
Proceedings of the fourth ACM international conference on Web search and data mining
An evaluation framework for plagiarism detection
COLING '10 Proceedings of the 23rd International Conference on Computational Linguistics: Posters
Designing incentives for inexpert human raters
Proceedings of the ACM 2011 conference on Computer supported cooperative work
The impact of social information on visual judgments
Proceedings of the SIGCHI Conference on Human Factors in Computing Systems
Design and implementation of relevance assessments using crowdsourcing
ECIR'11 Proceedings of the 33rd European conference on Advances in information retrieval
In search of quality in crowdsourcing for search engine evaluation
ECIR'11 Proceedings of the 33rd European conference on Advances in information retrieval
Crowdsourcing for book search evaluation: impact of hit design on comparative system ranking
Proceedings of the 34th international ACM SIGIR conference on Research and development in Information Retrieval
Repeatable and reliable search system evaluation using crowdsourcing
Proceedings of the 34th international ACM SIGIR conference on Research and development in Information Retrieval
Emerging theories and models of human computation systems: a brief survey
Proceedings of the 2nd international workshop on Ubiquitous crowdsouring
Capability-aligned matching: improving quality of games with a purpose
The 10th International Conference on Autonomous Agents and Multiagent Systems - Volume 2
Crowds in two seconds: enabling realtime crowd-powered interfaces
Proceedings of the 24th annual ACM symposium on User interface software and technology
Proceedings of the VLDB Endowment
Analyzing facebook privacy settings: user expectations vs. reality
Proceedings of the 2011 ACM SIGCOMM conference on Internet measurement conference
Proceedings of the Seventh Symposium on Usable Privacy and Security
Answering search queries with CrowdSearcher
Proceedings of the 21st international conference on World Wide Web
Improving the effectiveness of time-based display advertising
Proceedings of the 13th ACM Conference on Electronic Commerce
On aggregating labels from multiple crowd workers to infer relevance of documents
ECIR'12 Proceedings of the 34th European conference on Advances in Information Retrieval
Automatic identification of personal insults on social news sites
Journal of the American Society for Information Science and Technology
Reducing the barriers to writing verified specifications
Proceedings of the ACM international conference on Object oriented programming systems languages and applications
Proceedings of the 21st ACM international conference on Information and knowledge management
Extending search to crowds: a model-driven approach
Search Computing
Proceedings of the 2013 conference on Computer supported cooperative work
Perspectives on crowdsourcing annotations for natural language processing
Language Resources and Evaluation
An internet-scale idea generation system
ACM Transactions on Interactive Intelligent Systems (TiiS) - Special section on internet-scale human problem solving and regular papers
An introduction to crowdsourcing for language and multimedia technology research
PROMISE'12 Proceedings of the 2012 international conference on Information Retrieval Meets Information Visualization
Influence of timeline and named-entity components on user engagement
ECIR'13 Proceedings of the 35th European conference on Advances in Information Retrieval
An analysis of human factors and label accuracy in crowdsourcing relevance judgments
Information Retrieval
Implementing crowdsourcing-based relevance experimentation: an industrial perspective
Information Retrieval
Incentives and rewarding in social computing
Communications of the ACM
Proceedings of the 5th Annual ACM Web Science Conference
Experiences surveying the crowd: reflections on methods, participation, and reliability
Proceedings of the 5th Annual ACM Web Science Conference
Collabmap: crowdsourcing maps for emergency planning
Proceedings of the 5th Annual ACM Web Science Conference
Form digitization in BPO: from outsourcing to crowdsourcing?
Proceedings of the SIGCHI Conference on Human Factors in Computing Systems
Crowdsourcing performance evaluations of user interfaces
Proceedings of the SIGCHI Conference on Human Factors in Computing Systems
Analyzing crowd workers in mobile pay-for-answer q&a
Proceedings of the SIGCHI Conference on Human Factors in Computing Systems
Proceedings of the 22nd international conference on World Wide Web
Games with a Purpose or Mechanised Labour?: A Comparative Study
Proceedings of the 13th International Conference on Knowledge Management and Knowledge Technologies
Keep it simple: reward and task design in crowdsourcing
Proceedings of the Biannual Conference of the Italian Chapter of SIGCHI
From sensing to controlling: the state of the art in ubiquitous crowdsourcing
International Journal of Communication Networks and Distributed Systems
SRbench--a benchmark for soundtrack recommendation systems
Proceedings of the 22nd ACM international conference on Conference on information & knowledge management
Programming incentives in information systems
CAiSE'13 Proceedings of the 25th international conference on Advanced Information Systems Engineering
Feasibility of identifying eating moments from first-person images leveraging human computation
Proceedings of the 4th International SenseCam & Pervasive Imaging Conference
Repeatable and reliable semantic search evaluation
Web Semantics: Science, Services and Agents on the World Wide Web
The motivations and experiences of the on-demand mobile workforce
Proceedings of the 17th ACM conference on Computer supported cooperative work & social computing
Information extraction and manipulation threats in crowd-powered systems
Proceedings of the 17th ACM conference on Computer supported cooperative work & social computing
No "one-size fits all": towards a principled approach for incentives in mobile crowdsourcing
Proceedings of the 15th Workshop on Mobile Computing Systems and Applications
Newsvendor pull-to-center reconsidered
Decision Support Systems
Repeated labeling using multiple noisy labelers
Data Mining and Knowledge Discovery
Bucking the trend: improved evaluation and annotation practices for ESL error detection systems
Language Resources and Evaluation
| Hi-index | 0.03 |
The relationship between financial incentives and performance, long of interest to social scientists, has gained new relevance with the advent of web-based "crowd-sourcing" models of production. Here we investigate the effect of compensation on performance in the context of two experiments, conducted on Amazon's Mechanical Turk (AMT). We find that increased financial incentives increase the quantity, but not the quality, of work performed by participants, where the difference appears to be due to an "anchoring" effect: workers who were paid more also perceived the value of their work to be greater, and thus were no more motivated than workers paid less. In contrast with compensation levels, we find the details of the compensation scheme do matter---specifically, a "quota" system results in better work for less pay than an equivalent "piece rate" system. Although counterintuitive, these findings are consistent with previous laboratory studies, and may have real-world analogs as well.