Game theory and cost allocation problems
Management Science
Probabilistic reasoning in intelligent systems: networks of plausible inference
Probabilistic reasoning in intelligent systems: networks of plausible inference
Incentive compatible regression learning
Proceedings of the nineteenth annual ACM-SIAM symposium on Discrete algorithms
Power and stability in connectivity games
Proceedings of the 7th international joint conference on Autonomous agents and multiagent systems - Volume 2
Power in threshold network flow games
Autonomous Agents and Multi-Agent Systems
Power Indices in Spanning Connectivity Games
AAIM '09 Proceedings of the 5th International Conference on Algorithmic Aspects in Information and Management
The strategy-proofness landscape of merging
Journal of Artificial Intelligence Research
Approximating power indices: theoretical and empirical analysis
Autonomous Agents and Multi-Agent Systems
Probabilistic Graphical Models: Principles and Techniques - Adaptive Computation and Machine Learning
Bayesian knowledge corroboration with logical rules and user feedback
ECML PKDD'10 Proceedings of the 2010 European conference on Machine learning and knowledge discovery in databases: Part II
Crowd IQ: measuring the intelligence of crowdsourcing platforms
Proceedings of the 3rd Annual ACM Web Science Conference
Trust-based fusion of untrustworthy information in crowdsourcing applications
Proceedings of the 2013 international conference on Autonomous agents and multi-agent systems
Efficient budget allocation with accuracy guarantees for crowdsourcing classification tasks
Proceedings of the 2013 international conference on Autonomous agents and multi-agent systems
Characterizing and aggregating agent estimates
Proceedings of the 2013 international conference on Autonomous agents and multi-agent systems
Information verification during natural disasters
Proceedings of the 22nd international conference on World Wide Web companion
From blurry numbers to clear preferences: A mechanism to extract reputation in social networks
Expert Systems with Applications: An International Journal
Sharing rewards in cooperative connectivity games
Journal of Artificial Intelligence Research
| Hi-index | 0.00 |
We show how the quality of decisions based on the aggregated opinions of the crowd can be conveniently studied using a sample of individual responses to a standard IQ questionnaire. We aggregated the responses to the IQ questionnaire using simple majority voting and a machine learning approach based on a probabilistic graphical model. The score for the aggregated questionnaire, Crowd IQ, serves as a quality measure of decisions based on aggregating opinions, which also allows quantifying individual and crowd performance on the same scale. We show that Crowd IQ grows quickly with the size of the crowd but saturates, and that for small homogeneous crowds the Crowd IQ significantly exceeds the IQ of even their most intelligent member. We investigate alternative ways of aggregating the responses and the impact of the aggregation method on the resulting Crowd IQ. We also discuss Contextual IQ, a method of quantifying the individual participant's contribution to the Crowd IQ based on the Shapley value from cooperative game theory.