Detecting deception in reputation management
AAMAS '03 Proceedings of the second international joint conference on Autonomous agents and multiagent systems
Coping with inaccurate reputation sources: experimental analysis of a probabilistic trust model
Proceedings of the fourth international joint conference on Autonomous agents and multiagent systems
A survey of trust and reputation systems for online service provision
Decision Support Systems
Who are the crowdworkers?: shifting demographics in mechanical turk
CHI '10 Extended Abstracts on Human Factors in Computing Systems
Strategies for exploiting trust models in competitive multi-agent systems
MATES'09 Proceedings of the 7th German conference on Multiagent system technologies
Credibility: How Agents Can Handle Unfair Third-Party Testimonies in Computational Trust Models
IEEE Transactions on Knowledge and Data Engineering
The state of the art in trust and reputation systems: a framework for comparison
Journal of Theoretical and Applied Electronic Commerce Research
Analyzing the Amazon Mechanical Turk marketplace
XRDS: Crossroads, The ACM Magazine for Students - Comp-YOU-Ter
Exploration and Exploitation in Adaptive Trust-Based Decision Making in Dynamic Environments
WI-IAT '10 Proceedings of the 2010 IEEE/WIC/ACM International Conference on Web Intelligence and Intelligent Agent Technology - Volume 02
Crowdsourcing systems on the World-Wide Web
Communications of the ACM
Trust-based web service selection in virtual communities
Web Intelligence and Agent Systems
A reputation-aware decision-making approach for improving the efficiency of crowdsourcing systems
Proceedings of the 2013 international conference on Autonomous agents and multi-agent systems
A reputation management approach for resource constrained trustee agents
IJCAI'13 Proceedings of the Twenty-Third international joint conference on Artificial Intelligence
| Hi-index | 0.00 |
Crowd sourcing (CS) systems offer a new way for businesses and individuals to leverage on the power of mass collaboration to accomplish complex tasks in a divide-and-conquer manner. In existing CS systems, no facility has been provided for analyzing the trustworthiness of workers and providing decision support for allocating tasks to workers, which leads to high dependency of the quality of work on the behavior of workers in CS systems as shown in this paper. To address this problem, trust management mechanisms are urgently needed. Traditional trust management techniques are focused on identifying the most trustworthy service providers (SPs) as accurately as possible. Little thoughts were given to the question of how to utilize these SPs due to two common assumptions: 1) an SP can serve an unlimited number of requests in one time unit, and 2) a service consumer (SC) only needs to select one SP for interaction to complete a task. However, in CS systems, these two assumptions are no longer valid. Thus, existing models cannot be directly used for trust management in CS systems. This paper takes the first step towards a systematic investigation of trust management in CS systems by extending existing trust management models for CS trust management and conducting extensive experiments to study and analyze the performance of various trust management models in crowd sourcing. In this paper, the following key contributions are made. We 1) propose extensions to existing trust management approaches to enable them to operate in CS systems, 2) design a simulation test-bed based on the system characteristics of Amazon's Mechanical Turk (AMT) to make evaluation close to practical CS systems, 3) discuss the effect of incorporating trust management into CS system on the overall social welfare, and 4) identify the challenges and opportunities for future trust management research in CS systems.