Estimation of intrinsic image sequences from image+depth video

Authors:
Kyong Joon Lee;Qi Zhao;Xin Tong;Minmin Gong;Shahram Izadi;Sang Uk Lee;Ping Tan;Stephen Lin
Affiliations:
Microsoft Research Asia, China, Seoul National University, Korea;UC Santa Cruz;Microsoft Research Asia, China;Microsoft Research Asia, China;Microsoft Research Cambridge, UK;Seoul National University, Korea;National University of Singapore, Singapore;Microsoft Research Asia, China
Venue:
ECCV'12 Proceedings of the 12th European conference on Computer Vision - Volume Part VI
Year:
2012

Citing 11
Cited 1

Filters, Random Fields and Maximum Entropy (FRAME): Towards a Unified Theory for Texture Modeling

International Journal of Computer Vision
Learning Low-Level Vision

International Journal of Computer Vision - Special issue on statistical and computational theories of vision: modeling, learning, sampling and computing, Part I
A Variational Framework for Retinex

International Journal of Computer Vision
Recovering Shading from Color Images

ECCV '92 Proceedings of the Second European Conference on Computer Vision
Recovering Intrinsic Images from a Single Image

IEEE Transactions on Pattern Analysis and Machine Intelligence
Edge Suppression by Gradient Field Transformation Using Cross-Projection Tensors

CVPR '06 Proceedings of the 2006 IEEE Computer Society Conference on Computer Vision and Pattern Recognition - Volume 2
Intrinsic colorization

ACM SIGGRAPH Asia 2008 papers
User-assisted intrinsic images

ACM SIGGRAPH Asia 2009 papers
Illumination normalization with time-dependent intrinsic images for video surveillance

CVPR'03 Proceedings of the 2003 IEEE computer society conference on Computer vision and pattern recognition
KinectFusion: real-time 3D reconstruction and interaction using a moving depth camera

Proceedings of the 24th annual ACM symposium on User interface software and technology
A Closed-Form Solution to Retinex with Nonlocal Texture Constraints

IEEE Transactions on Pattern Analysis and Machine Intelligence

Coherent intrinsic images from photo collections

ACM Transactions on Graphics (TOG) - Proceedings of ACM SIGGRAPH Asia 2012

Quantified Score

Hi-index	0.00

Visualization

Abstract

We present a technique for estimating intrinsic images from image+depth video, such as that acquired from a Kinect camera. Intrinsic image decomposition in this context has importance in applications like object modeling, in which surface colors need to be recovered without illumination effects. The proposed method is based on two new types of decomposition constraints derived from the multiple viewpoints and reconstructed 3D scene geometry of the video data. The first type provides shading constraints that enforce relationships among the shading components of different surface points according to their similarity in surface orientation. The second type imposes temporal constraints that favor consistency in the intrinsic color of a surface point seen in different video frames, which improves decomposition in cases of view-dependent non-Lambertian reflections. Local and non-local variants of the two constraints are employed in a manner complementary to local and non-local reflectance constraints used in previous works. Together they are formulated within a linear system that allows for efficient optimization. Experimental results demonstrate that each of the new constraints appreciably elevates the quality of intrinsic image estimation, and that they jointly yield decompositions that compare favorably to current techniques.