Virtualized Reality: Constructing Virtual Worlds from Real Scenes

• Authors: Takeo Kanade;Peter Rander;P. J. Narayanan
• Venue: IEEE MultiMedia
• Year: 1997

Virtualized Reality is a new visual medium that immerses viewers in a virtual reconstruction of real-world events, i.e., a virtualized model of the real world. This medium addresses two fundamental limitations of other visual media such as motion pictures and video: fixed viewpoints and two-dimensional views. Virtualized Reality enables users to select their own viewpoints at view time, independent of the actual camera positions used to capture the event. It also provides stereoscopic reconstructions of the event, enabling a viewer to be completely immersed in the virtualized event. The Virtualized Reality world model consists of real images and depth information computed from these images. The depth information is converted into 3D triangle meshes representing the surfaces visible in each image. Virtual camera images are generated by rendering the 3D meshes into the virtual camera, mapping the real images onto the meshes as texture. The virtual images, therefore, preserve the visual realism of even complex scenes. In contrast, typical virtual worlds are built by hand or using CAD tools. They tend to be simple and often lack visual realism. Realistic training in a virtualized work space is an important application area of Virtualized Reality. For example, surgical training could be enhanced by enabling the trainees to explore the virtualized operation at their will. A new medium of entertainment can also develop from Virtualized Reality. This medium would allow a viewer to watch a ballet performance seated on the edge of the stage or a basketball game while standing on the court or running with a particular player. Readers may contact Rander at the Robotics Institute, Carnegie Mellon University, Smith Hall, Pittsburgh, PA 15213-3890, e-mail [e-mail].