evaluating the stability and credibility of ontology matching methods

  • Authors:
  • Xing Niu;Haofen Wang;Gang Wu;Guilin Qi;Yong Yu

  • Affiliations:
  • Shanghai Jiao Tong University;Shanghai Jiao Tong University;Northeastern University;Southeast University;Shanghai Jiao Tong University

  • Venue:
  • ESWC'11 Proceedings of the 8th extended semantic web conference on The semantic web: research and applications - Volume Part I
  • Year:
  • 2011

Quantified Score

Hi-index 0.00

Visualization

Abstract

Ontology matching is one of the key research topics in Semantic Web. In the last few years, many matching methods have been proposed to generate matches between different ontologies either automatically or semi-automatically. To select appropriate ones, users need some measures to judge whether a method can achieve the similar compliance even on one dataset without reference matches and whether such a method is reliable w.r.t. its output result along with the confidence. However, widely-used traditional measures like precision and recall fail to provide sufficient hints. In this paper, we design two novel evaluation measures to evaluate stability of matching methods and one measure to evaluate credibility of matching confidence values, which help answer the above two questions. Additionally, we carry out comparisons among several carefully selected methods systematically using our new measures. Besides, we report some interesting findings such as identifying potential defects of our subjects.