Infrastructure and workflow for the formal evaluation of semantic search technologies

  • Authors:
  • Stuart N. Wrigley;Raúl García-Castro;Cássia Trojahn

  • Affiliations:
  • University of Sheffield, Sheffield, United Kingdom;Universidad Politécnica de Madrid, Madrid, Spain;INRIA & LIG, Montbonnot Saint Martin, France

  • Venue:
  • Proceedings of the 2011 workshop on Data infrastructurEs for supporting information retrieval evaluation
  • Year:
  • 2011

Quantified Score

Hi-index 0.00

Visualization

Abstract

This paper describes an infrastructure for the automated evaluation of semantic technologies and, in particular, semantic search technologies. For this purpose, we present an evaluation framework which follows a service-oriented approach for evaluating semantic technologies and uses the Business Process Execution Language (BPEL) to define evaluation workflows that can be executed by process engines. This framework supports a variety of evaluations, from different semantic areas, including search, and is extendible to new evaluations. We show how BPEL addresses this diversity as well as how it is used to solve specific challenges such as heterogeneity, error handling and reuse.