Performance measures for object detection evaluation

Authors:
Bahadır Özdemir;Selim Aksoy;Sandra Eckert;Martino Pesaresi;Daniele Ehrlich
Affiliations:
Department of Computer Engineering, Bilkent University, Ankara 06800, Turkey;Department of Computer Engineering, Bilkent University, Ankara 06800, Turkey;Institute for the Protection and Security of the Citizen, European Commission, Joint Research Centre, 21020 Ispra (VA), Italy;Institute for the Protection and Security of the Citizen, European Commission, Joint Research Centre, 21020 Ispra (VA), Italy;Institute for the Protection and Security of the Citizen, European Commission, Joint Research Centre, 21020 Ispra (VA), Italy
Venue:
Pattern Recognition Letters
Year:
2010

Citing 11
Cited 3

An Experimental Comparison of Range Image Segmentation Algorithms

IEEE Transactions on Pattern Analysis and Machine Intelligence
The Earth Mover's Distance as a Metric for Image Retrieval

International Journal of Computer Vision
Quantitative methods of evaluating image segmentation

ICIP '95 Proceedings of the 1995 International Conference on Image Processing (Vol. 3)-Volume 3 - Volume 3
Performance Evaluation of Object Detection Algorithms

ICPR '02 Proceedings of the 16 th International Conference on Pattern Recognition (ICPR'02) Volume 3 - Volume 3
Algorithm Performance Contest

ICPR '00 Proceedings of the International Conference on Pattern Recognition - Volume 4
Learning to Detect Natural Image Boundaries Using Local Brightness, Color, and Texture Cues

IEEE Transactions on Pattern Analysis and Machine Intelligence
On the Use of Error Propagation for Statistical Validation of Computer Vision Software

IEEE Transactions on Pattern Analysis and Machine Intelligence
Evaluation campaigns and TRECVid

MIR '06 Proceedings of the 8th ACM international workshop on Multimedia information retrieval
On the use of the overlapping area matrix for image segmentation evaluation: A survey and new performance measures

Pattern Recognition Letters
Performance characterization in computer vision: A guide to best practices

Computer Vision and Image Understanding
The 2005 PASCAL visual object classes challenge

MLCW'05 Proceedings of the First international conference on Machine Learning Challenges: evaluating Predictive Uncertainty Visual Object Classification, and Recognizing Textual Entailment

Extraction of complex patterns from multiresolution remote sensing images: A hierarchical top-down methodology

Pattern Recognition
Spatio-temporal reasoning for the classification of satellite image time series

Pattern Recognition Letters
Unsupervised segmentation and classification of cervical cell images

Pattern Recognition

Quantified Score

Hi-index	0.11

Visualization

Abstract

We propose a new procedure for quantitative evaluation of object detection algorithms. The procedure consists of a matching stage for finding correspondences between reference and output objects, an accuracy score that is sensitive to object shapes as well as boundary and fragmentation errors, and a ranking step for final ordering of the algorithms using multiple performance indicators. The procedure is illustrated on a building detection task where the resulting rankings are consistent with the visual inspection of the detection maps.