A protocol for performance evaluation of line detection algorithms
Machine Vision and Applications - Special issue on performance evaluation
Empirical Performance Evaluation of Graphics Recognition Systems
IEEE Transactions on Pattern Analysis and Machine Intelligence
A Tabular Survey of Automated Table Processing
GREC '99 Selected Papers from the Third International Workshop on Graphics Recognition, Recent Advances
Exploiting WWW Resources in Experimental Document Analysis Research
DAS '02 Proceedings of the 5th International Workshop on Document Analysis Systems V
Applying the T-Recs Table Recognition System to the Business Letter Domain
ICDAR '01 Proceedings of the Sixth International Conference on Document Analysis and Recognition
Why Table Ground-Truthing is Hard
ICDAR '01 Proceedings of the Sixth International Conference on Document Analysis and Recognition
Evaluating Document Analysis Results via Graph Probing
ICDAR '01 Proceedings of the Sixth International Conference on Document Analysis and Recognition
Tabular abstraction, editing, and formatting
Tabular abstraction, editing, and formatting
Document structure analysis and performance evaluation
Document structure analysis and performance evaluation
A survey of table recognition: Models, observations, transformations, and inferences
International Journal on Document Analysis and Recognition
A language for specifying and comparing table recognition strategies
A language for specifying and comparing table recognition strategies
Historical Recall and Precision: Summarizing Generated Hypotheses
ICDAR '05 Proceedings of the Eighth International Conference on Document Analysis and Recognition
The TXL source transformation language
Science of Computer Programming - The fourth workshop on language descriptions, tools, and applications (LDTA'04)
Hi-index | 0.02 |
Traditionally computer vision and pattern recognition algorithms are evaluated by measuring differences between final interpretations and ground truth. These black-box evaluations ignore intermediate results, making it difficult to use intermediate results in diagnosing errors and optimization. We propose "opening the box," representing vision algorithms as sequences of decision points where recognition results are selected from a set of alternatives. For this purpose, we present a domain-specific language for pattern recognition tasks, the Recognition Strategy Language (RSL). At run-time, an RSL interpreter records a complete history of decisions made during recognition, as it applies them to a set of interpretations maintained for the algorithm. Decision histories provide a rich new source of information: recognition errors may be traced back to the specific decisions that caused them, and intermediate interpretations may be recovered and displayed. This additional information also permits new evaluation metrics that include false negatives (correct hypotheses that the algorithm generates and later rejects), such as the percentage of ground truth hypotheses generated (historical recall ), and the percentage of generated hypotheses that are correct(historical precision). We illustrate the approach through an analysis of cell detection in two published table recognition algorithms.