Validating an Automated Evaluation Procedure for Ontology Triples in the Privacy Domain

  • Authors:
  • Peter Spyns;Giles Hogben

  • Affiliations:
  • Vrije Universiteit Brussel-STAR Lab, Belgium;European Commission Joint Research Centre-IPSC, Ispra, Italy

  • Venue:
  • Proceedings of the 2005 conference on Legal Knowledge and Information Systems: JURIX 2005: The Eighteenth Annual Conference
  • Year:
  • 2005

Quantified Score

Hi-index 0.01

Visualization

Abstract

In this paper we validate a simple method to objectively assess the results of extracting material (c.q. triples) from text corpora to build ontologies. The EU Privacy Directive has been used as corpus. Two domain experts have manually validated the results. Several experimental settings have been tried. As the evaluation scores are rather modest (sensitivity or recall: 0.5, specificity: 0.539 and precision: 0.21), we see them as a baseline reference for future experiments. Nevertheless, the human experts appreciate the automated evaluation procedure as sufficiently effective and time-saving for usage in real-life ontology modelling situations.