ADiEU: Toward Domain-Based Evaluation of Spoken Dialog Systems

  • Authors:
  • Jan Kleindienst;Jan Cuřín;Martin Labský

  • Affiliations:
  • IBM Research, Prague, Czech Republic;IBM Research, Prague, Czech Republic;IBM Research, Prague, Czech Republic

  • Venue:
  • Proceedings of the 13th International Conference on Human-Computer Interaction. Part I: New Trends
  • Year:
  • 2009

Quantified Score

Hi-index 0.00

Visualization

Abstract

We propose a new approach toward evaluation of spoken dialog systems. The novelty of our method is based on utilization of domain-specific knowledge combined with the deterministic measurement of dialog system performance on a set of individual tasks within the domain. The proposed methodology thus attempts to answer questions such as: "How well is my dialog system performing on a specific domain?", "How much has my dialog system improved since the previous version?", "How much is my dialog system better/worse than other dialog systems performing on that domain?"