The evaluation of legal knowledge based systems

  • Authors:
  • Andrew Stranieri;John Zeleznikow

  • Affiliations:
  • School of Information Technology and Mathematical Sciences, University of Ballarat, Ballarat, Victoria, Australia;Database Research Laboratory, Applied Computing Research Institute, La Trobe University, Bundoora, Victoria, Australia, 3083

  • Venue:
  • ICAIL '99 Proceedings of the 7th international conference on Artificial intelligence and law
  • Year:
  • 1999

Quantified Score

Hi-index 0.00

Visualization

Abstract

Evaluation strategies to assess the effectiveness of legal knowledge based systems enable strengths and limitations of systems to be accurately articulated. This facilitates efforts in the research community to develop systems and also promotes the adoption of research prototypes in the commercial world. However, evaluation strategies for systems that operate in a domain as complex as law are difficult to specify. In this paper, we present an evaluation framework put forward by Reich and describe how this motivated the evaluation of our systems in Australian family law. Strategies surveyed include a comparison of linear regression with neural networks, user acceptance surveys, a comparison of system predictions with those from past cases, and a comparison of system outputs with those proposed by a panel of lawyers. Specific criteria for the evaluation of explanation facilities are also described.