An Experiment in Comparing Human-Computation Techniques

  • Authors:
  • Stefan Thaler;Elena Simperl;Stephan Wolger

  • Affiliations:
  • University of Innsbruck;Karlsruhe Institute of Technology;University of Innsbruck

  • Venue:
  • IEEE Internet Computing
  • Year:
  • 2012

Quantified Score

Hi-index 0.00

Visualization

Abstract

Human computation can address complex computational problems by tapping into large resource pools for relatively little cost. Two prominent human-computation techniques — games with a purpose (GWAP) and microtask crowdsourcing — can help resolve semantic-technology-related tasks, including knowledge representation, ontology alignment, and semantic annotation. To evaluate which approach is better with respect to costs and benefits, the authors employ categorization challenges in Wikipedia to ultimately create a large, general-purpose ontology. They first use the OntoPronto GWAP, then replicate its problem-solving setting in Amazon Mechanical Turk, using a similar task-design structure, evaluation mechanisms, and input data.