Painterly animation using video semantics and feature correspondence

  • Authors:

  • Liang Lin;Kun Zeng;Han Lv;Yizhou Wang;Yingqing Xu;Song-Chun Zhu

  • Affiliations:

  • Lotus Hill Research Institute, China and Sun Yat-Sen University, Guangzhou, China;Lotus Hill Research Institute, China;Lotus Hill Research Institute, China;Peking University, Beijing, China;Microsoft Research Asia, Beijing, China;Lotus Hill Research Institute, China and University of California, Los Angeles

  • Venue:
  • NPAR '10 Proceedings of the 8th International Symposium on Non-Photorealistic Animation and Rendering
  • Year:
  • 2010

Quantified Score

Hi-index 0.00

Visualization

Abstract

We present an interactive system that stylizes an input video into a painterly animation. The system consists of two phases. The first is an Video Parsing phase that extracts and labels semantic objects with different material properties (skin, hair, cloth, and so on) in the video, and then establishes robust correspondence between frames for discriminative image features inside each object. The second Painterly Rendering phase performs the stylization based on the video semantics and feature correspondence. Compared to the previous work, the proposed method advances painterly animation in three aspects: Firstly, we render artistic painterly styles using a rich set of example-based brush strokes. These strokes, placed in multiple layers and passes, are automatically selected according to the video semantics. Secondly, we warp brush strokes according to global object deformations, so that the strokes appear to be tightly attached to the object surfaces. Thirdly, we propose a series of novel teniques to reduce the scintillation effects. Results applying our system to several video clips show that it produces expressive oil painting animations.