Correspondence as energy-based segmentation

Authors:
Stanley T. Birchfield;Braga Natarajan;Carlo Tomasi
Affiliations:
Department of Electrical and Computer Engineering, Clemson University, Clemson, SC 29634, USA;Department of Electrical and Computer Engineering, Clemson University, Clemson, SC 29634, USA;Department of Computer Science, Duke University, Durham, NC 27708, USA
Venue:
Image and Vision Computing
Year:
2007

Citing 20
Cited 0

Visual reconstruction

Visual reconstruction
Occlusions and binocular stereo

International Journal of Computer Vision
Normalized Cuts and Image Segmentation

IEEE Transactions on Pattern Analysis and Machine Intelligence
Fast Approximate Energy Minimization via Graph Cuts

IEEE Transactions on Pattern Analysis and Machine Intelligence
A Taxonomy and Evaluation of Dense Two-Frame Stereo Correspondence Algorithms

International Journal of Computer Vision
M2Tracker: A Multi-View Approach to Segmenting and Tracking People in a Cluttered Scene

International Journal of Computer Vision
Cooperative Robust Estimation Using Layers of Support

IEEE Transactions on Pattern Analysis and Machine Intelligence
Disparity-Space Images and Large Occlusion Stereo

ECCV '94 Proceedings of the Third European Conference-Volume II on Computer Vision - Volume II
Multi-camera Scene Reconstruction via Graph Cuts

ECCV '02 Proceedings of the 7th European Conference on Computer Vision-Part III
Projective registration with difference decomposition

CVPR '97 Proceedings of the 1997 Conference on Computer Vision and Pattern Recognition (CVPR '97)
A Unified Mixture Framework for Motion Segmentation: Incorporating Spatial Coherence and Estimating the Number of Models

CVPR '96 Proceedings of the 1996 Conference on Computer Vision and Pattern Recognition (CVPR '96)
Markov Random Fields with Efficient Approximations

CVPR '98 Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition
Layered representation of motion video using robust maximum-likelihood estimation of mixture models and MDL encoding

ICCV '95 Proceedings of the Fifth International Conference on Computer Vision
A Maximum-Flow Formulation of the N-Camera Stereo Correspondence Problem

ICCV '98 Proceedings of the Sixth International Conference on Computer Vision
Visual Correspondence Using Energy Minimization and Mutual Information

ICCV '03 Proceedings of the Ninth IEEE International Conference on Computer Vision - Volume 2
What Energy Functions Can Be Minimizedvia Graph Cuts?

IEEE Transactions on Pattern Analysis and Machine Intelligence
Spatially coherent clustering using graph cuts

CVPR'04 Proceedings of the 2004 IEEE computer society conference on Computer vision and pattern recognition
Surfaces with occlusions from layered stereo

CVPR'03 Proceedings of the 2003 IEEE computer society conference on Computer vision and pattern recognition
Stereo by Intra- and Inter-Scanline Search Using Dynamic Programming

IEEE Transactions on Pattern Analysis and Machine Intelligence
Representing moving images with layers

IEEE Transactions on Image Processing

Quantified Score

Hi-index	0.00

Visualization

Abstract

We pose the correspondence problem as one of energy-based segmentation. In this framework, correspondence assigns each pixel in an image to exactly one of several non-overlapping regions, and it also computes a displacement function for each region. The framework is better able to capture the scene geometry than the more direct formulation of matching pixels in two or more images, particularly when the surfaces in the scene are not fronto-parallel. To illustrate the framework, we present a specific correspondence algorithm that minimizes an energy functional by alternating between (1) segmenting the image into a number of non-overlapping regions using the multiway-cut algorithm of Boykov, Veksler, and Zabih; and (2) finding the affine parameters describing the displacement of the pixels in each region. After convergence, a final step escapes local minima due to over-segmentation. The basic algorithm is extended in two ways: using ground control points to detect long, thin regions; and warping segmentation results to efficiently process image sequences. Experiments on real images show the algorithm's ability to find an accurate segmentation and displacement map, as well as discontinuities and creases, on a wide variety of stereo and motion imagery.