Learning complex concepts using crowdsourcing: a Bayesian approach

Authors:
Paolo Viappiani;Sandra Zilles;Howard J. Hamilton;Craig Boutilier
Affiliations:
Department of Computer Science, Aalborg University, Denmark;Department of Computer Science, University of Regina, Canada;Department of Computer Science, University of Regina, Canada;Department of Computer Science, University of Toronto, Canada
Venue:
ADT'11 Proceedings of the Second international conference on Algorithmic decision theory
Year:
2011

Citing 8
Cited 0

Learning in the presence of malicious errors

SIAM Journal on Computing
Introduction to Reinforcement Learning

Introduction to Reinforcement Learning
An Approximate Nonmyopic Computation for Value of Information

IEEE Transactions on Pattern Analysis and Machine Intelligence
Learning Conjunctive Concepts in Structural Domains

Machine Learning
Queries and Concept Learning

Machine Learning
Queries and Concept Learning

Machine Learning
Online feature elicitation in interactive optimization

ICML '09 Proceedings of the 26th Annual International Conference on Machine Learning
Version spaces: a candidate elimination approach to rule learning

IJCAI'77 Proceedings of the 5th international joint conference on Artificial intelligence - Volume 1

Quantified Score

Hi-index	0.01

Visualization

Abstract

We develop a Bayesian approach to concept learning for crowdsourcing applications. A probabilistic belief over possible concept definitions is maintained and updated according to (noisy) observations from experts, whose behaviors are modeled using discrete types. We propose recommendation techniques, inference methods, and query selection strategies to assist a user charged with choosing a configuration that satisfies some (partially known) concept. Our model is able to simultaneously learn the concept definition and the types of the experts. We evaluate our model with simulations, showing that our Bayesian strategies are effective even in large concept spaces with many uninformative experts.