Unsupervised skeleton extraction and motion capture from 3D deformable matching

Authors:
Quanshi Zhang;Xuan Song;Xiaowei Shao;Ryosuke Shibasaki;Huijing Zhao
Affiliations:
Center for Spatial Information Science, University of Tokyo, Japan;Center for Spatial Information Science, University of Tokyo, Japan;Center for Spatial Information Science, University of Tokyo, Japan;Center for Spatial Information Science, University of Tokyo, Japan;Key Laboratory of Machine Perception (MoE), Peking University, China
Venue:
Neurocomputing
Year:
2013

Citing 19
Cited 1

Unsupervised Learning of Human Motion

IEEE Transactions on Pattern Analysis and Machine Intelligence
Skeletal Parameter Estimation from Optical Motion Capture Data

CVPR '05 Proceedings of the 2005 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'05) - Volume 2 - Volume 02
Articulated Structure from Motion by Factorization

CVPR '05 Proceedings of the 2005 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'05) - Volume 2 - Volume 02
Automatic Kinematic Chain Building from Feature Trajectories of Articulated Objects

CVPR '06 Proceedings of the 2006 IEEE Computer Society Conference on Computer Vision and Pattern Recognition - Volume 1
Building Models of Animals from Video

IEEE Transactions on Pattern Analysis and Machine Intelligence
Convergent Tree-Reweighted Message Passing for Energy Minimization

IEEE Transactions on Pattern Analysis and Machine Intelligence
Harmonic skeleton for realistic character animation

SCA '07 Proceedings of the 2007 ACM SIGGRAPH/Eurographics symposium on Computer animation
Example-based skeleton extraction

SGP '07 Proceedings of the fifth Eurographics symposium on Geometry processing
Skeleton extraction by mesh contraction

ACM SIGGRAPH 2008 papers
A Factorization-Based Approach for Articulated Nonrigid Shape, Motion and Kinematic Chain Recovery From Video

IEEE Transactions on Pattern Analysis and Machine Intelligence
Efficient MRF deformation model for non-rigid image matching

Computer Vision and Image Understanding
Unsupervised Learning of Skeletons from Motion

ECCV '08 Proceedings of the 10th European Conference on Computer Vision: Part III
Feature Correspondence Via Graph Matching: Models and Global Optimization

ECCV '08 Proceedings of the 10th European Conference on Computer Vision: Part II
Curve skeleton extraction from incomplete point cloud

ACM SIGGRAPH 2009 papers
Learning kinematic models for articulated objects

IJCAI'09 Proceedings of the 21st international jont conference on Artifical intelligence
Learning Articulated Structure and Motion

International Journal of Computer Vision
Shape-from-silhouette of articulated objects and its use for human body kinematics estimation and motion capture

CVPR'03 Proceedings of the 2003 IEEE computer society conference on Computer vision and pattern recognition
Real-time human pose recognition in parts from single depth images

CVPR '11 Proceedings of the 2011 IEEE Conference on Computer Vision and Pattern Recognition
Delaunay conforming iso-surface, skeleton extraction and noise removal

Computational Geometry: Theory and Applications

Unsupervised human skeleton extraction from Kinect depth images

Proceedings of the 4th International Conference on Internet Multimedia Computing and Service

Quantified Score

Hi-index	0.01

Visualization

Abstract

This paper presents a novel method to extract skeletons of complex articulated objects from 3D point cloud sequences collected by the Kinect. Our approach is more robust than the traditional video-based and stereo-based approaches, as the Kinect directly provides 3D information without any markers, 2D-to-3D-transition assumptions, and feature point extraction. We track all the raw 3D points on the object, and utilize the point trajectories to determine the object skeleton. The point tracking is achieved by the 3D non-rigid matching based on the Markov Random Field (MRF) Deformation Model. To reduce the large computational cost of the non-rigid matching, a coarse-to-fine procedure is proposed. To the best of our knowledge, this is the first to extract skeletons of highly deformable objects from 3D point cloud sequences by point tracking. Experiments prove our method's good performance, and the extracted skeletons are successfully applied to the motion capture.