Using motion capture for real-time augmented reality scenes

• Authors: Hiromitsu Sato;Michael Cohen
• Venue: Proceedings of the 13th International Conference on Humans and Computers
• Year: 2010

The purpose of this study is improvement of real-time human-computer interface in using augmented reality. We propose animating an avatar in real-time with motion capture system and augmented reality. Development of ubiquitous computing is impressive in recent years. Augmented reality is an instance of ubiquitous computing technology. Augmented reality is related to virtual reality, characterized by composition of real and virtual, and display of electronic information as virtual object and giving the real world addition of information. Because computer performance is continually upgrading, we can display complicated 3D models, and it is possible to easily add information to a computer such as a mobile telephone. Core technologies of augmented reality are position recognition, image recognition, and orientation recognition. In the past, high performance machines were needed to process these. Using modern hardware, we can process such data using mobile phone as well as notebook. Recently, Sekai Camera became famous as augmented reality software that runs on iPhone and Android. Assuming continual computer performance upgrade, there will appear a lot of augmented reality software we can use in the future. However, the means to affect information in augmented reality have not fully developed yet. It is reasonable to suppose that we can use motion capture as technique to let a real human affect a figurative 3D object in augmented reality. Motion capture typically involves tracking parts of the body by infra-red cameras to obtain streamed position data. We can realize real human movement by applying this data to a 3D model. Development of devices and technology to capture motion advances. Reasonable devices and software have become widely used. For example, Kinect on the input device for Xbox 360, sold since 2010 by Microsoft, allows manipulation of a 3D model on the computer without a controller. In the future, techniques to handle 3D objects will grow more popular. The merit of both augmented reality and motion capture is to enable dynamic manipulation a virtual space. It is likely that one will be able to exchange one's own information for another's in asymptotically realistic environments. An avatar was produced which works on augmented reality in real-time as proof of concept. When it was developed, efficiency was emphasized by using motion capture software "Vicon IQ," motion capture real-time engine "Tarsus," C language library "ARToolKit," and 3DCG software "3ds max."