Truthing, Testing and Evaluation Issues in Complex Systems

Authors:
Alfred Lawson
Affiliations:
-
Venue:
ICDAR '01 Proceedings of the Sixth International Conference on Document Analysis and Recognition
Year:
2001

Citing 0
Cited 3

A soft computing model for mapping incomplete/approximate postal addresses to mail delivery points

Applied Soft Computing
A fuzzy symbolic inference system for postal address component extraction and labelling

FSKD'06 Proceedings of the Third international conference on Fuzzy Systems and Knowledge Discovery
Symbolic data structure for postal address representation and address validation through symbolic knowledge base

PReMI'05 Proceedings of the First international conference on Pattern Recognition and Machine Intelligence

Quantified Score

Hi-index	0.00

Visualization

Abstract

Abstract: This paper describes the issues involved in the design of a system for evaluating improvements in the performance of a real-time address recognition system being used by the United States Postal Service for processing mail-piece images. Evaluation of the performance of recognition systems is normally carried out by measuring the performance of the system on a representative sample of images. Designing a comprehensive and valid testing scenario is a complex task that requires careful attention. Sampling live mail-stream to generate a deck of images representative of the general mail-stream for testing, truthing (generating reference data on a significant number of images), grading and evaluation, and designing tools to facilitate these functions are important topics that need to be addressed. This paper describes the efforts of the United States Postal Service and CEDAR towards developing an infra-structure for sampling, truthing and testing of mailstream images.