Distilling the wisdom of crowds: weighted aggregation of decisions on multiple issues

Authors:
Eyal Baharad;Jacob Goldberger;Moshe Koppel;Shmuel Nitzan
Affiliations:
Department of Economics, University of Haifa, Haifa, Israel 31905;School of Engineering, Bar Ilan University, Ramat Gan, Israel 52900;Department of Computer Sciences, Bar Ilan University, Ramat Gan, Israel 52900;Department of Economics, Bar Ilan University, Ramat Gan, Israel 52900
Venue:
Autonomous Agents and Multi-Agent Systems
Year:
2011

Citing 0
Cited 2

Using the crowd for top-k and group-by queries

Proceedings of the 16th International Conference on Database Theory
A voice in the crowd: Broader implications for crowdsourcing translation during crisis

Journal of Information Science

Quantified Score

Hi-index	0.00

Visualization

Abstract

Given the judgments of multiple voters regarding some issue, it is generally assumed that the best way to arrive at some collective judgment is by following the majority. We consider here the now common case in which each voter expresses some (binary) judgment regarding each of a multiplicity of independent issues and assume that each voter has some fixed (unknown) probability of making a correct judgment for any given issue. We leverage the fact that multiple votes by each voter are known in order to demonstrate, both analytically and empirically, that a method based on maximum likelihood estimation is superior to the simple majority rule for arriving at true collective judgments.