Acknowledging insufficiency in the evaluation of legal knowledge-based systems: strategies towards a broadbased evaluation model

  • Authors:

  • Maria Jean J. Hall;John Zeleznikow

  • Affiliations:

  • Donald Berman Laboratory for Information Technology and Law, Applied Computing Research Institute, La Trobe University Bundoora, Vic., Australia 3083;Donald Berman Laboratory for Information Technology and Law, Applied Computing Research Institute, La Trobe University Bundoora, Vic., Australia 3083

  • Venue:
  • Proceedings of the 8th international conference on Artificial intelligence and law
  • Year:
  • 2001

Quantified Score

Hi-index 0.00

Visualization

Abstract

This paper considers the need for evaluation of knowledge-based systems in general and legal knowledge-based systems in particular. Some special features of legal knowledge-based systems pertinent to their evaluation are presented. The expected benefits of such evaluations are discussed. and some of the difficulties likely to be encountered are outlined.The proceedings of four International Conferences on Artificial Intelligence and Law are analysed to determine the rate of reporting evaluations in non-theoretical papers. These papers had a low rate of consideration of evaluation issues reflecting common practice in research biased development environments. These results confirm that more attention to evaluation is needed in the legal knowledge based systems domain.This paper foreshadows the development of an evaluation methodology tailored specifically for legal knowledge-based systems. Evaluation strategies beyond verification and validation are drawn upon, both from the international ISO/IEC 14598 and 9126 standards and also from previous work on evaluation models for knowledge-based systems.