Historical Recall and Precision: Summarizing Generated Hypotheses
ICDAR '05 Proceedings of the Eighth International Conference on Document Analysis and Recognition
The Recognition Strategy Language
ICDAR '05 Proceedings of the Eighth International Conference on Document Analysis and Recognition
White-Box Evaluation of Computer Vision Algorithms through Explicit Decision-Making
ICVS '09 Proceedings of the 7th International Conference on Computer Vision Systems: Computer Vision Systems
Recognition tasks are imitation games
ICAPR'05 Proceedings of the Third international conference on Advances in Pattern Recognition - Volume Part I
| Hi-index | 0.01 |
Table recognition algorithms may be described by models of table location and structure, and decisions made relative to these models. These algorithms are usually defined informally as a sequence of decisions with supporting data observations and transformations. In this investigation, we formalize these algorithms as strategies in an imitation game, where the goal of the game is to match table interpretations from a chosen procedure as closely as possible. The chosen procedure may be a person or persons producing ‘ground truth,’ or an algorithm. To describe table recognition strategies we have defined the Recognition Strategy Language (RSL). RSL is a simple functional language for describing strategies as sequences of abstract decision types whose results are determined by any suit able decision method. RSL defines and maintains interpretation trees, a simple data structure for describing recognition results. For each interpretation in an interpretation tree, we annotate hypothesis histories which capture the creation, revision, and rejection of individual hypotheses, such as the logical type and structure of regions. We present a proof-of-concept using two strategies from the literature. We demonstrate how RSL allows strategies to be specified at the level of decisions rather than algorithms, and we compare results of our strategy implementations using new techniques. In particular, we introduce historical recall and precision metrics. Conventional recall and precision characterize hypotheses accepted after a strategy has finished. Historical recall and precision provide additional information by describing all generated hypotheses, including any rejected in the final result.