Video enhancement leveraging high-quality depth maps

  • Authors:
  • Zhongmou Cai;Enhua Wu

  • Affiliations:
  • University of Macau, Macao, China;University of Macau, Macao, China, and Institute of Software, Chinese Academy of Sciences, Beijing, China

  • Venue:
  • Proceedings of the 11th ACM SIGGRAPH International Conference on Virtual-Reality Continuum and its Applications in Industry
  • Year:
  • 2012

Quantified Score

Hi-index 0.00

Visualization

Abstract

Using an image-based rendering (IBR) algorithm, we can reconstruct and enhance a low-quality video by extracting colors from several high-quality photographs according to the correspondences. In this paper, we present several improvements for the IBR algorithm to improve the reconstruction faithfulness and temporal coherence of the resultant video. Concretely, we propose to use another state-of-the-art multiview stereo (MVS) method to recover more accurate and highly consistent depths for the video, and adapt it to handle the sparse photographs. Leveraging the consistent video depths, we further propose a simple but effective method to generate more faithful photograph candidates, which also helps to maintain the temporal coherence. The effectiveness of our improvements is verified by achieving the effect of video super-resolution. Besides, we improve the structure-from-motion (SfM) system to process the heterogeneous dataset containing several unordered photographs and a video clip more efficiently for our application.