Two variations on ontology alignment evaluation: methodological issues

Authors:
Laura Hollink;Mark Van Assem;Shenghui Wang;Antoine Isaac;Guus Schreiber
Affiliations:
Department of Computer Science, Vrije Universiteit Amsterdam, The Netherlands;Department of Computer Science, Vrije Universiteit Amsterdam, The Netherlands;Department of Computer Science, Vrije Universiteit Amsterdam, The Netherlands;Department of Computer Science, Vrije Universiteit Amsterdam, The Netherlands;Department of Computer Science, Vrije Universiteit Amsterdam, The Netherlands
Venue:
ESWC'08 Proceedings of the 5th European semantic web conference on The semantic web: research and applications
Year:
2008

Citing 7
Cited 4

Using graded relevance assessments in IR evaluation

Journal of the American Society for Information Science and Technology
Ontology mapping: the state of the art

The Knowledge Engineering Review
Ontology Matching

Ontology Matching
Semantic precision and recall for ontology alignment evaluation

IJCAI'07 Proceedings of the 20th international joint conference on Artifical intelligence
Discovering simple mappings between relational database schemas and ontologies

ISWC'07/ASWC'07 Proceedings of the 6th international The semantic web and 2nd Asian conference on Asian semantic web conference
MultimediaN e-culture demonstrator

ISWC'06 Proceedings of the 5th international conference on The Semantic Web
A survey of schema-based matching approaches

Journal on Data Semantics IV

Matching large ontologies: A divide-and-conquer approach

Data & Knowledge Engineering
Building a global normalized ontology for integrating geographic data sources

Computers & Geosciences
Ontology alignment evaluation initiative: six years of experience

Journal on data semantics XV
A configurable translation-based cross-lingual ontology mapping system to adjust mapping outcomes

Web Semantics: Science, Services and Agents on the World Wide Web

Quantified Score

Hi-index	0.00

Visualization

Abstract

Evaluation of ontology alignments is in practice done in two ways: (1) assessing individual correspondences and (2) comparing the alignment to a reference alignment. However, this type of evaluation does not guarantee that an application which uses the alignment will perform well. In this paper, we contribute to the current ontology alignment evaluation practices by proposing two alternative evaluation methods that take into account some characteristics of a usage scenario without doing a full-fledged end-to-end evaluation. We compare different evaluation approaches in three case studies, focussing on methodological issues. Each case study considers an alignment between a different pair of ontologies, ranging from rich and well-structured to small and poorly structured. This enables us to conclude on the use of different evaluation approaches in different settings.