Getting more from segmentation evaluation

Authors:
Martin Scaiano;Diana Inkpen
Affiliations:
University of Ottawa, Ottawa, ON, Canada;University of Ottawa, Ottawa, ON, Canada
Venue:
NAACL HLT '12 Proceedings of the 2012 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies
Year:
2012

Citing 8
Cited 1

A critique and improvement of an evaluation metric for text segmentation

Computational Linguistics
Spoken and written news story segmentation using lexical chains

NAACLstudent '03 Proceedings of the 2003 Conference of the North American Chapter of the Association for Computational Linguistics on Human Language Technology: Proceedings of the HLT-NAACL 2003 student research workshop - Volume 3
Minimum cut model for spoken lecture segmentation

ACL-44 Proceedings of the 21st International Conference on Computational Linguistics and the 44th annual meeting of the Association for Computational Linguistics
User-oriented text segmentation evaluation measure

SIGIR '07 Proceedings of the 30th annual international ACM SIGIR conference on Research and development in information retrieval
Movie scene segmentation using background information

Pattern Recognition
On Evaluation Methodologies for Text Segmentation Algorithms

ICTAI '07 Proceedings of the 19th IEEE International Conference on Tools with Artificial Intelligence - Volume 02
Revealing the structure of medical dictations with conditional random fields

EMNLP '08 Proceedings of the Conference on Empirical Methods in Natural Language Processing
An analysis of quantitative aspects in the evaluation of thematic segmentation algorithms

SigDIAL '06 Proceedings of the 7th SIGdial Workshop on Discourse and Dialogue

Topical segmentation: a study of human performance and a new measure of quality

NAACL HLT '12 Proceedings of the 2012 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies

Quantified Score

Hi-index	0.00

Visualization

Abstract

We introduce a new segmentation evaluation measure, WinPR, which resolves some of the limitations of WindowDiff. WinPR distinguishes between false positive and false negative errors; produces more intuitive measures, such as precision, recall, and F-measure; is insensitive to window size, which allows us to customize near miss sensitivity; and is based on counting errors not windows, but still provides partial reward for near misses.