SIAM Journal on Discrete Mathematics
Computers and Intractability: A Guide to the Theory of NP-Completeness
Computers and Intractability: A Guide to the Theory of NP-Completeness
Expertise identification using email communications
CIKM '03 Proceedings of the twelfth international conference on Information and knowledge management
The Wisdom of Crowds
CarTel: a distributed mobile sensor computing system
Proceedings of the 4th international conference on Embedded networked sensor systems
Nericell: rich monitoring of road and traffic conditions using mobile smartphones
Proceedings of the 6th ACM conference on Embedded network sensor systems
Crowdsourcing for relevance evaluation
ACM SIGIR Forum
Finding a team of experts in social networks
Proceedings of the 15th ACM SIGKDD international conference on Knowledge discovery and data mining
Cheap and fast---but is it good?: evaluating non-expert annotations for natural language tasks
EMNLP '08 Proceedings of the Conference on Empirical Methods in Natural Language Processing
On the "localness" of user-generated content
Proceedings of the 2010 ACM conference on Computer supported cooperative work
Toward trustworthy mobile sensing
Proceedings of the Eleventh Workshop on Mobile Computing Systems & Applications
CrowdSearch: exploiting crowds for accurate real-time image search on mobile phones
Proceedings of the 8th international conference on Mobile systems, applications, and services
Quality management on Amazon Mechanical Turk
Proceedings of the ACM SIGKDD Workshop on Human Computation
Towards trustworthy participatory sensing
HotSec'09 Proceedings of the 4th USENIX conference on Hot topics in security
The Journal of Machine Learning Research
Location-based crowdsourcing: extending crowdsourcing to the real world
Proceedings of the 6th Nordic Conference on Human-Computer Interaction: Extending Boundaries
Crowdsourcing systems on the World-Wide Web
Communications of the ACM
CrowdDB: answering queries with crowdsourcing
Proceedings of the 2011 ACM SIGMOD International Conference on Management of data
Design and Analysis of Approximation Algorithms
Design and Analysis of Approximation Algorithms
Whom to ask?: jury selection for decision making tasks on micro-blog services
Proceedings of the VLDB Endowment
GeoCrowd: enabling query answering with spatial crowdsourcing
Proceedings of the 20th International Conference on Advances in Geographic Information Systems
Hi-index | 0.00 |
With the abundance and ubiquity of mobile devices, a new class of applications, called spatial crowdsourcing, is emerging, which enables spatial tasks (i.e., tasks related to a location) assigned to and performed by human workers. However, one of the major challenges with spatial crowdsourcing is how to verify the validity of the results provided by workers, when the workers are not trusted equally. To tackle this problem, we assume every worker has a reputation score, which states the probability that the worker performs a task correctly. Moreover, we define a confidence level for every spatial task, which states that the answer to the given spatial task is only accepted if its confidence is higher than a certain threshold. Thus, the problem we are trying to solve is to maximize the number of spatial tasks that are assigned to a set of workers while satisfying the confidence levels of those tasks. Note that a unique aspect of our problem is that the optimal assignment of tasks heavily depends on the geographical locations of workers and tasks. This means that every spatial task should be assigned to enough number of workers such that their aggregate reputation satisfies the confidence of the task. Consequently, an exhaustive approach needs to compute the aggregate reputation score (using a typical decision fusion aggregation mechanism, such as voting) for all possible subsets of the workers, which renders the problem complex (we show it is NP-hard). Subsequently, we propose a number of heuristics and utilizing real-world and synthetic data in extensive sets of experiments we show that we can achieve close to optimal performance with the cost of a greedy approach, by exploiting our problem's unique characteristics.