Utility evaluation of cross-document information extraction

Authors:
Heng Ji;Zheng Chen;Jonathan Feldman;Antonio Gonzalez;Ralph Grishman;Vivek Upadhyay
Affiliations:
City University of New York, New York, NY;City University of New York, New York, NY;City University of New York, New York, NY;City University of New York, New York, NY;New York University, New York, NY;City University of New York, New York, NY
Venue:
HLT '10 Human Language Technologies: The 2010 Annual Conference of the North American Chapter of the Association for Computational Linguistics
Year:
2010

Citing 4
Cited 0

Remote usability testing

interactions
Usability Engineering

Usability Engineering
TAP-XL: an automated analyst's assistant

NAACL-Demonstrations '03 Proceedings of the 2003 Conference of the North American Chapter of the Association for Computational Linguistics on Human Language Technology: Demonstrations - Volume 4
Cross-document temporal and spatial person tracking system demonstration

NAACL-Demonstrations '09 Proceedings of Human Language Technologies: The 2009 Annual Conference of the North American Chapter of the Association for Computational Linguistics, Companion Volume: Demonstration Session

Quantified Score

Hi-index	0.00

Visualization

Abstract

We describe a utility evaluation to determine whether cross-document information extraction (IE) techniques measurably improve user performance in news summary writing. Two groups of subjects were asked to perform the same time-restricted summary writing tasks, reading news under different conditions: with no IE results at all, with traditional single-document IE results, and with cross-document IE results. Our results show that, in comparison to using source documents only, the quality of summary reports assembled using IE results, especially from cross-document IE, was significantly better and user satisfaction was higher. We also compare the impact of different user groups on the results.