Video completion via motion guided spatial-temporal global optimization

  • Authors:
  • Ming Liu;Shifeng Chen;Jianzhuang Liu;Xiaoou Tang

  • Affiliations:
  • The Chinese University of Hong Kong, Hong Kong, Hong Kong;Shenzhen Institute of Advanced Technology, Chinese Academy of Sciences, Shenzhen, China;The Chinese University of Hong Kong/ Shenzhen Institute of Advanced Technology, Chinese Academy of Sciences, Hong Kong/ Shenzhen, China;The Chinese University of Hong Kong/ Shenzhen Institute of Advanced Technology, Chinese Academy of Sciences, Hong Kong/ Shenzhen, China

  • Venue:
  • MM '09 Proceedings of the 17th ACM international conference on Multimedia
  • Year:
  • 2009

Quantified Score

Hi-index 0.00

Visualization

Abstract

In this paper, a novel global optimization based approach is proposed for video completion whose target is to restore the spatial-temporal missing regions of a video in a visually plausible way. Our algorithm consists of two stages: motion field completion and color completion via global optimization. First, local motions within the missing parts are completed patch-by-patch greedily using pre-computed available motions in the video. Then the missing regions are filled by sampling patches from available parts of the video. We formulate the video completion as a global energy minimization problem by Markov random fields (MRFs). Based on the completed motion field of the video, a well-defined energy function involving both spatial and temporal coherence relationship is constructed. A coarse-to-fine Belief Propagation (BP) is proposed to solve the optimization problem. Experimental results have demonstrated the good performance of our algorithm.