IPSN '08 Proceedings of the 7th international conference on Information processing in sensor networks
Designing games with a purpose
Communications of the ACM - Designing games with a purpose
Get another label? improving data quality and data mining using multiple, noisy labelers
Proceedings of the 14th ACM SIGKDD international conference on Knowledge discovery and data mining
Cheap and fast---but is it good?: evaluating non-expert annotations for natural language tasks
EMNLP '08 Proceedings of the Conference on Empirical Methods in Natural Language Processing
Analyzing the Amazon Mechanical Turk marketplace
XRDS: Crossroads, The ACM Magazine for Students - Comp-YOU-Ter
A Bayesian approach to learning Bayesian networks with local structure
UAI'97 Proceedings of the Thirteenth conference on Uncertainty in artificial intelligence
Bandit based monte-carlo planning
ECML'06 Proceedings of the 17th European conference on Machine Learning
Incentives for truthful reporting in crowdsourcing
Proceedings of the 11th International Conference on Autonomous Agents and Multiagent Systems - Volume 3
Pairwise ranking aggregation in a crowdsourced setting
Proceedings of the sixth ACM international conference on Web search and data mining
Proceedings of the 2013 conference on Computer supported cooperative work
It is about time: time aware quality management for interactive systems with humans in the loop
CHI '13 Extended Abstracts on Human Factors in Computing Systems
Light at the end of the tunnel: a Monte Carlo approach to computing value of information
Proceedings of the 2013 international conference on Autonomous agents and multi-agent systems
Trust-based fusion of untrustworthy information in crowdsourcing applications
Proceedings of the 2013 international conference on Autonomous agents and multi-agent systems
Efficient budget allocation with accuracy guarantees for crowdsourcing classification tasks
Proceedings of the 2013 international conference on Autonomous agents and multi-agent systems
Characterizing and aggregating agent estimates
Proceedings of the 2013 international conference on Autonomous agents and multi-agent systems
A lightweight combinatorial approach for inferring the ground truth from multiple annotators
MLDM'13 Proceedings of the 9th international conference on Machine Learning and Data Mining in Pattern Recognition
Lifelong learning for acquiring the wisdom of the crowd
IJCAI'13 Proceedings of the Twenty-Third international joint conference on Artificial Intelligence
Look versus leap: computing value of information with high-dimensional streaming evidence
IJCAI'13 Proceedings of the Twenty-Third international joint conference on Artificial Intelligence
STFU NOOB!: predicting crowdsourced decisions on toxic behavior in online games
Proceedings of the 23rd international conference on World wide web
Quizz: targeted crowdsourcing with a billion (potential) users
Proceedings of the 23rd international conference on World wide web
Community-based bayesian aggregation models for crowdsourcing
Proceedings of the 23rd international conference on World wide web
Hi-index | 0.00 |
We show how machine learning and inference can be harnessed to leverage the complementary strengths of humans and computational agents to solve crowdsourcing tasks. We construct a set of Bayesian predictive models from data and describe how the models operate within an overall crowd-sourcing architecture that combines the efforts of people and machine vision on the task of classifying celestial bodies defined within a citizens' science project named Galaxy Zoo. We show how learned probabilistic models can be used to fuse human and machine contributions and to predict the behaviors of workers. We employ multiple inferences in concert to guide decisions on hiring and routing workers to tasks so as to maximize the efficiency of large-scale crowdsourcing processes based on expected utility.