Benchmarking federated SPARQL query engines: are existing testbeds enough?

Authors:
Gabriela Montoya;Maria-Esther Vidal;Oscar Corcho;Edna Ruckhaus;Carlos Buil-Aranda
Affiliations:
Universidad Simón Bolívar, Venezuela;Universidad Simón Bolívar, Venezuela;Ontology Engineering Group, Universidad Politécnica de Madrid, Spain;Universidad Simón Bolívar, Venezuela;Department of Computer Science, Pontificia Universidad Católica, Chile
Venue:
ISWC'12 Proceedings of the 11th international conference on The Semantic Web - Volume Part II
Year:
2012

Citing 8
Cited 0

Semantics and complexity of SPARQL

ACM Transactions on Database Systems (TODS)
Querying distributed RDF data sources with SPARQL

ESWC'08 Proceedings of the 5th European semantic web conference on The semantic web: research and applications
Foundations of SPARQL query optimization

Proceedings of the 13th International Conference on Database Theory
Semantics and optimization of the SPARQL 1.1 federation extension

ESWC'11 Proceedings of the 8th extended semantic web conference on The semanic web: research and applications - Volume Part II
ANAPSID: an adaptive query processing engine for SPARQL endpoints

ISWC'11 Proceedings of the 10th international conference on The semantic web - Volume Part I
FedBench: a benchmark suite for federated semantic data query processing

ISWC'11 Proceedings of the 10th international conference on The semantic web - Volume Part I
FedX: optimization techniques for federated query processing on linked data

ISWC'11 Proceedings of the 10th international conference on The semantic web - Volume Part I
ADERIS: an adaptive query processor for joining federated SPARQL endpoints

OTM'11 Proceedings of the 2011th Confederated international conference on On the move to meaningful internet systems - Volume Part II

Quantified Score

Hi-index	0.00

Visualization

Abstract

Testbeds proposed so far to evaluate, compare, and eventually improve SPARQL query federation systems have still some limitations. Some variables and configurations that may have an impact on the behavior of these systems (e.g., network latency, data partitioning and query properties) are not sufficiently defined; this affects the results and repeatability of independent evaluation studies, and hence the insights that can be obtained from them. In this paper we evaluate FedBench, the most comprehensive testbed up to now, and empirically probe the need of considering additional dimensions and variables. The evaluation has been conducted on three SPARQL query federation systems, and the analysis of these results has allowed to uncover properties of these systems that would normally be hidden with the original testbeds.