An Automatic Closed-Loop Methodology for Generating Character Groundtruth for Scanned Documents
IEEE Transactions on Pattern Analysis and Machine Intelligence
Towards robust features for classifying audio in the CueVideo system
MULTIMEDIA '99 Proceedings of the seventh ACM international conference on Multimedia (Part 1)
Using Diagram Generation Software to Improve Diagram Recognition: A Case Study of Music Notation
IEEE Transactions on Pattern Analysis and Machine Intelligence
Optical Character Recognition: An Illustrated Guide to the Frontier
Optical Character Recognition: An Illustrated Guide to the Frontier
Structured Document Image Analysis
Structured Document Image Analysis
The Second International Graphics Recognition Contest - Raster to Vector Conversion: A Report
GREC '97 Selected Papers from the Second International Workshop on Graphics Recognition, Algorithms and Systems
Toward a Common Validation Methodology for Segmentation and Registration Algorithms
MICCAI '00 Proceedings of the Third International Conference on Medical Image Computing and Computer-Assisted Intervention
ICDAR '01 Proceedings of the Sixth International Conference on Document Analysis and Recognition
Why Table Ground-Truthing is Hard
ICDAR '01 Proceedings of the Sixth International Conference on Document Analysis and Recognition
Tabular abstraction, editing, and formatting
Tabular abstraction, editing, and formatting
The TeXbook
Extraction, layout analysis and classification of diagrams in PDF documents
ICDAR '03 Proceedings of the Seventh International Conference on Document Analysis and Recognition - Volume 2
Building Synthetic Graphical Documents for Performance Evaluation
Graphics Recognition. Recent Advances and New Opportunities
Tools for monitoring, visualizing, and refining collections of noisy documents
Proceedings of The Third Workshop on Analytics for Noisy Unstructured Text Data
From Tessellations to Table Interpretation
Calculemus '09/MKM '09 Proceedings of the 16th Symposium, 8th International Conference. Held as Part of CICM '09 on Intelligent Computer Mathematics
A platform for storing, visualizing, and interpreting collections of noisy documents
AND '10 Proceedings of the fourth workshop on Analytics for noisy unstructured text data
Recognition tasks are imitation games
ICAPR'05 Proceedings of the Third international conference on Advances in Pattern Recognition - Volume Part I
| Hi-index | 0.00 |
We examine the nature of ground-truth: whether it is always well-defined fora given task, oron ly relative and approximate. In the conventional scenario, reference data is produced by recording the interpretation of each test document using a chosen data-entry platform. Looking a little more closely at this process, we study its constituents and their interrelations. We provide examples from the literature and from our own experiments where non-trivial problems with each of the components appear to preclude the possibility of real progress in evaluating automated graphics recognition systems, and propose possible solutions. More specifically, for documents with complex structure we recommend multi-valued, layered, weighted, functional ground-truth supported by model-guided reference data-entry systems and protocols. Mostly, however, we raise far more questions than we currently have answers for.