PARADISE-style evaluation of a human-human library corpus

  • Authors:
  • Rebecca J. Passonneau;Irene Alvarado;Phil Crone;Simon Jerome

  • Affiliations:
  • Columbia University;Columbia University;Columbia University;Columbia University

  • Venue:
  • SIGDIAL '11 Proceedings of the SIGDIAL 2011 Conference
  • Year:
  • 2011

Quantified Score

Hi-index 0.00

Visualization

Abstract

We apply a PARADISE-style evaluation to a human-human dialogue corpus that was collected to support the design of a spoken dialogue system for library transactions. The book request dialogue task we investigate is informational in nature: a book request is considered successful if the librarian is able to identify a specific book for the patron. PARADISE assumes that user satisfaction can be modeled as a regression over task success and dialogue costs. The PARADISE model we derive includes features that characterize two types of qualitative features. The first has to do with the specificity of the communicative goals, given a request for an item. The second has to do with the number and location of overlapping turns, which can sometimes signal rapport between the speakers.