Information Retrieval
Binarization of Low Quality Text Using a Markov Random Field Model
ICPR '02 Proceedings of the 16 th International Conference on Pattern Recognition (ICPR'02) Volume 3 - Volume 3
Incorporating non-local information into information extraction systems by Gibbs sampling
ACL '05 Proceedings of the 43rd Annual Meeting on Association for Computational Linguistics
A general framework for the evaluation of symbol recognition methods
International Journal on Document Analysis and Recognition
An Overview of the Tesseract OCR Engine
ICDAR '07 Proceedings of the Ninth International Conference on Document Analysis and Recognition - Volume 02
Pattern Recognition Methods for Querying and Browsing Technical Documentation
CIARP '08 Proceedings of the 13th Iberoamerican congress on Pattern Recognition: Progress in Pattern Recognition, Image Analysis and Applications
Results of the RIMES Evaluation Campaign for Handwritten Mail Processing
ICDAR '09 Proceedings of the 2009 10th International Conference on Document Analysis and Recognition
Relying on topic subsets for system ranking estimation
Proceedings of the 18th ACM conference on Information and knowledge management
ICDAR '11 Proceedings of the 2011 International Conference on Document Analysis and Recognition
A probabilistic interpretation of precision, recall and F-score, with implication for evaluation
ECIR'05 Proceedings of the 27th European conference on Advances in Information Retrieval Research
Ground truth for layout analysis performance evaluation
DAS'06 Proceedings of the 7th international conference on Document Analysis Systems
Spatio-structural symbol description with statistical feature add-on
GREC'11 Proceedings of the 9th international conference on Graphics Recognition: new trends and challenges
| Hi-index | 0.00 |
In this paper we present a way to use precision and recall measures in total absence of ground truth. We develop a probabilistic interpretation of both measures and show that, provided a sufficient number of data sources are available, it offers a viable performance measure to compare methods if no ground truth is available. This paper also shows the limitations of the approach, in case a systematic bias is present in all compared methods, but shows that it maintains a very high level of overall coherence and stability. It opens broader perspectives and can be extended to handling partial or unreliable ground truth, as well as levels of prior confidence in the methods it aims to compare.