Validating the TEMAA LE evaluation methodology: a case study on Danish spelling checkers

  • Authors:
  • Patrizia Paggio;Nancy L. Underwood

  • Affiliations:
  • Center for Sprogteknologi, Njalsgade 80, 2300 Copenhagen S, Denmark. Email: patrizia@cst.ku.dk;Center for Sprogteknologi, Njalsgade 80, 2300 Copenhagen S, Denmark. E-mail: nancy@cst.ku.dk

  • Venue:
  • Natural Language Engineering
  • Year:
  • 1998

Quantified Score

Hi-index 0.00

Visualization

Abstract

This paper describes the work carried out at the Center for Sprogteknologi in Copenhagen to validate the LE evaluation methodology developed by the LRE project TEMAA. TEMAA has developed a framework for the evaluation of LE products, implemented in a Parameterisable Testbed (PTB). The framework allows for a modular, formal and exible description of user requirements and objects of evaluation, it accommodates test methods of various kind and provides a methodology for assessing test results in the light of the requirements expressed by different user types. While the fundamentals of the TEMAA framework are meant to apply to adequacy evaluation of LE products in general, a detailed methodology has been worked out for the evaluation of spelling and grammar checkers, and applied to the concrete evaluation of Danish and Italian spelling checkers. The main focus of this paper is on showing that the general methodology provides a valid model for designing and carrying out a concrete evaluation, as in the case study on Danish spelling checkers.