Interactive music video application for smartphones based on free-viewpoint video and audio rendering

Authors:
Toshiharu Horiuchi;Hiroshi Sankoh;Tsuneo Kato;Sei Naito
Affiliations:
KDDI R&D Laboratories, Inc., Fujimino, Saitama, Japan;KDDI R&D Laboratories, Inc., Fujimino, Saitama, Japan;KDDI R&D Laboratories, Inc., Fujimino, Saitama, Japan;KDDI R&D Laboratories, Inc., Fujimino, Saitama, Japan
Venue:
Proceedings of the 20th ACM international conference on Multimedia
Year:
2012

Citing 2
Cited 0

Virtualized Reality: Constructing Virtual Worlds from Real Scenes

IEEE MultiMedia
Free viewpoint video generation for walk-through experience using image-based rendering

MM '08 Proceedings of the 16th ACM international conference on Multimedia

Quantified Score

Hi-index	0.00

Visualization

Abstract

This paper presents a novel interactive music video application for smartphones based on free-viewpoint video technology in conjunction with three-dimensional positional audio technology. A user can enjoy a music video from a moving viewpoint that the user can manipulate by the touch screen, with the positional audio through the headphone. A user can even manipulate the positions of the performers on the stage as well as the viewpoint. The application, consisting of our audio rendering engine for multiple AAC ADTS files and our video rendering engine for multiple H.264 ES files, runs on a smartphone in stand-alone mode. The application has been released as official content from a music label for Android and iOS.