An experimental evaluation of the assumption of independence in multiversion programming
IEEE Transactions on Software Engineering
Data Diversity: An Approach to Software Fault Tolerance
IEEE Transactions on Computers - Fault-Tolerant Computing
Specification-based test oracles for reactive systems
ICSE '92 Proceedings of the 14th international conference on Software engineering
Pseudo-oracles for non-testable programs
ACM '81 Proceedings of the ACM '81 conference
Investigating the use of analysis contracts to improve the testability of object-oriented code
Software—Practice & Experience
Benchmarking Attribute Selection Techniques for Discrete Class Data Mining
IEEE Transactions on Knowledge and Data Engineering
Digital Geometry: Geometric Methods for Digital Picture Analysis
Digital Geometry: Geometric Methods for Digital Picture Analysis
Image Processing - Principles and Applications
Image Processing - Principles and Applications
Meta-Evaluation of Image Segmentation Using Machine Learning
CVPR '06 Proceedings of the 2006 IEEE Computer Society Conference on Computer Vision and Pattern Recognition - Volume 1
Random Walks for Image Segmentation
IEEE Transactions on Pattern Analysis and Machine Intelligence
Image Processing in Radiology: Current Applications (Medical Radiology / Diagnostic Imaging)
Image Processing in Radiology: Current Applications (Medical Radiology / Diagnostic Imaging)
Data Mining: Practical Machine Learning Tools and Techniques, Second Edition (Morgan Kaufmann Series in Data Management Systems)
The Oracle Problem for Testing against Quantified Properties
QSIC '07 Proceedings of the Seventh International Conference on Quality Software
Introduction to Software Testing
Introduction to Software Testing
Journal of Systems and Software
Automatic system testing of programs without test oracles
Proceedings of the eighteenth international symposium on Software testing and analysis
Evaluating classifiers by means of test data with noisy labels
IJCAI'03 Proceedings of the 18th international joint conference on Artificial intelligence
MICCAI'07 Proceedings of the 10th international conference on Medical image computing and computer-assisted intervention - Volume Part I
Foundations of Software Testing
Foundations of Software Testing
Evaluating segmentation error without ground truth
MICCAI'12 Proceedings of the 15th international conference on Medical Image Computing and Computer-Assisted Intervention - Volume Part I
Hi-index | 0.00 |
An image segmentation algorithm delineates (an) object(s) of interest in an image. Its output is referred to as a segmentation. Developing these algorithms is a manual, iterative process involving repetitive verification and validation tasks. This process is time-consuming and depends on the availability of experts, who may be a scarce resource (e.g., medical experts). We propose a framework referred to as Image Segmentation Automated Oracle (ISAO) that uses machine learning to construct an oracle, which can then be used to automatically verify the correctness of image segmentations, thus saving substantial resources and making the image segmentation verification and validation task significantly more efficient. The framework also gives informative feedback to the developer as the segmentation algorithm evolves and provides a systematic means of testing different parametric configurations of the algorithm. During the initial learning phase, segmentations from the first few (optimally two) versions of the segmentation algorithm are manually verified by experts. The similarity of successive segmentations of the same images is also measured in various ways. This information is then fed to a machine learning algorithm to construct a classifier that distinguishes between consistent and inconsistent segmentation pairs (as determined by an expert) based on the values of the similarity measures associated with each segmentation pair. Once the accuracy of the classifier is deemed satisfactory to support a consistency determination, the classifier is then used to determine whether the segmentations that are produced by subsequent versions of the algorithm under test, are (in)consistent with already verified segmentations from previous versions. This information is then used to automatically draw conclusions about the correctness of the segmentations. We have successfully applied this approach to 3D segmentations of the cardiac left ventricle obtained from CT scans and have obtained promising results (accuracies of 95%). Even though more experiments are needed to quantify the effectiveness of the approach in real-world applications, ISAO shows promise in increasing the quality and testing efficiency of image segmentation algorithms.