Balancing the Role of Priors in Multi-Observer Segmentation Evaluation

  • Authors:

  • Yaoyao Zhu;Xiaolei Huang;Wei Wang;Daniel Lopresti;Rodney Long;Sameer Antani;Zhiyun Xue;George Thoma

  • Affiliations:

  • Department of Computer Science and Engineering, Lehigh University, Bethlehem, USA 18015;Department of Computer Science and Engineering, Lehigh University, Bethlehem, USA 18015;Department of Computer Science and Engineering, Lehigh University, Bethlehem, USA 18015;Department of Computer Science and Engineering, Lehigh University, Bethlehem, USA 18015;National Library of Medicine, National Institutes of Health, Bethesda, USA 20894;National Library of Medicine, National Institutes of Health, Bethesda, USA 20894;National Library of Medicine, National Institutes of Health, Bethesda, USA 20894;National Library of Medicine, National Institutes of Health, Bethesda, USA 20894

  • Venue:
  • Journal of Signal Processing Systems
  • Year:
  • 2009

Quantified Score

Hi-index 0.00

Visualization

Abstract

Comparison of a group of multiple observer segmentations is known to be a challenging problem. A good segmentation evaluation method would allow different segmentations not only to be compared, but to be combined to generate a "true" segmentation with higher consensus. Numerous multi-observer segmentation evaluation approaches have been proposed in the literature, and STAPLE in particular probabilistically estimates the true segmentation by optimal combination of observed segmentations and a prior model of the truth. An Expectation---Maximization (EM) algorithm, STAPLE's convergence to the desired local minima depends on good initializations for the truth prior and the observer-performance prior. However, accurate modeling of the initial truth prior is nontrivial. Moreover, among the two priors, the truth prior always dominates so that in certain scenarios when meaningful observer-performance priors are available, STAPLE can not take advantage of that information. In this paper, we propose a Bayesian decision formulation of the problem that permits the two types of prior knowledge to be integrated in a complementary manner in four cases with differing application purposes: (1) with known truth prior; (2) with observer prior; (3) with neither truth prior nor observer prior; and (4) with both truth prior and observer prior. The third and fourth cases are not discussed (or effectively ignored) by STAPLE, and in our research we propose a new method to combine multiple-observer segmentations based on the maximum a posterior (MAP) principle, which respects the observer prior regardless of the availability of the truth prior. Based on the four scenarios, we have developed a web-based software application that implements the flexible segmentation evaluation framework for digitized uterine cervix images. Experiment results show that our framework has flexibility in effectively integrating different priors for multi-observer segmentation evaluation and it also generates results comparing favorably to those by the STAPLE algorithm and the Majority Vote Rule.