KinectFusion: Real-time dense surface mapping and tracking

Authors:
Richard A. Newcombe;Shahram Izadi;Otmar Hilliges;David Molyneaux;David Kim;Andrew J. Davison;Pushmeet Kohli;Jamie Shotton;Steve Hodges;Andrew Fitzgibbon
Affiliations:
Imperial College London, UK;Microsoft Research, UK;Microsoft Research, UK;Microsoft Research, Lancaster University, UK;Microsoft Research, Newcastle University, UK;Imperial College London, UK;Microsoft Research, UK;Microsoft Research, UK;Microsoft Research, UK;Microsoft Research, UK
Venue:
ISMAR '11 Proceedings of the 2011 10th IEEE International Symposium on Mixed and Augmented Reality
Year:
2011

Citing 0
Cited 58

KinectFusion: real-time 3D reconstruction and interaction using a moving depth camera

Proceedings of the 24th annual ACM symposium on User interface software and technology
Tuning stereo image matching with stereo video sequence processing

Proceedings of the 2012 Joint International Conference on Human-Centered Computer Environments
Real-time computer vision with OpenCV

Communications of the ACM
Realtime Computer Vision with OpenCV

Queue - Processors
Increasing kinect application development productivity by an enhanced hardware abstraction

Proceedings of the 4th ACM SIGCHI symposium on Engineering interactive computing systems
Computer-Aided design of tactile models: taxonomy and case studies

ICCHP'12 Proceedings of the 13th international conference on Computers Helping People with Special Needs - Volume Part II
Walk&Sketch: create floor plans with an RGB-D camera

Proceedings of the 2012 ACM Conference on Ubiquitous Computing
Integrating the physical environment into mobile remote collaboration

MobileHCI '12 Proceedings of the 14th international conference on Human-computer interaction with mobile devices and services
Augmented Reality: Reciprocal shading for mixed reality

Computers and Graphics
Learning to efficiently detect repeatable interest points in depth data

ECCV'12 Proceedings of the 12th European conference on Computer Vision - Volume Part I
A generative model for online depth fusion

ECCV'12 Proceedings of the 12th European conference on Computer Vision - Volume Part V
Real-Time camera tracking: when is high frame-rate best?

ECCV'12 Proceedings of the 12th European conference on Computer Vision - Volume Part VII
Technical demonstration on model based training, detection and pose estimation of texture-less 3d objects in heavily cluttered scenes

ECCV'12 Proceedings of the 12th international conference on Computer Vision - Volume Part III
A unified energy minimization framework for model fitting in depth

ECCV'12 Proceedings of the 12th international conference on Computer Vision - Volume 2
OmniKinect: real-time dense volumetric data acquisition and applications

Proceedings of the 18th ACM symposium on Virtual reality software and technology
Integrating approximate depth data into dense image correspondence estimation

Proceedings of the 9th European Conference on Visual Media Production
Vote based correspondence for 3D point-set registration

Proceedings of the Eighth Indian Conference on Computer Vision, Graphics and Image Processing
A pipeline for building 3D models using depth cameras

Proceedings of the Eighth Indian Conference on Computer Vision, Graphics and Image Processing
Seamless texture map generation from multiple images

Proceedings of the 27th Conference on Image and Vision Computing New Zealand
Using the Kinect as a navigation sensor for mobile robotics

Proceedings of the 27th Conference on Image and Vision Computing New Zealand
A memory-efficient kinectfusion using octree

CVM'12 Proceedings of the First international conference on Computational Visual Media
OctoMap: an efficient probabilistic 3D mapping framework based on octrees

Autonomous Robots
3D building reconstruction and thermal mapping in fire brigade operations

Proceedings of the 4th Augmented Human International Conference
Octree-based fusion for realtime 3D reconstruction

Graphical Models
Dense scene reconstruction with points of interest

ACM Transactions on Graphics (TOG) - SIGGRAPH 2013 Conference Proceedings
Scalable real-time volumetric surface reconstruction

ACM Transactions on Graphics (TOG) - SIGGRAPH 2013 Conference Proceedings
Model based training, detection and pose estimation of texture-less 3d objects in heavily cluttered scenes

ACCV'12 Proceedings of the 11th Asian conference on Computer Vision - Volume Part I
Fusion of time-of-flight and stereo for disambiguation of depth measurements

ACCV'12 Proceedings of the 11th Asian conference on Computer Vision - Volume Part IV
Exploring high-level plane primitives for indoor 3d reconstruction with a hand-held RGB-D camera

ACCV'12 Proceedings of the 11th international conference on Computer Vision - Volume 2
KinectAvatar: fully automatic body capture using a single kinect

ACCV'12 Proceedings of the 11th international conference on Computer Vision - Volume 2
Spin image revisited: fast candidate selection using outlier forest search

ACCV'12 Proceedings of the 11th international conference on Computer Vision - Volume 2
Vision: mapping the world in 3d through first-person vision devices with mercator

Proceeding of the fourth ACM workshop on Mobile cloud computing and services
Dynamic 2D/3D registration for the Kinect

ACM SIGGRAPH 2013 Courses
Real-time 3D reconstruction at scale using voxel hashing

ACM Transactions on Graphics (TOG)
3D self-portraits

ACM Transactions on Graphics (TOG)
Real-time 3D gaze analysis in mobile applications

Proceedings of the 2013 Conference on Eye Tracking South Africa
Simultaneous localization and mapping for event-based vision systems

ICVS'13 Proceedings of the 9th international conference on Computer Vision Systems
Quick and dirty: streamlined 3D scanning in archaeology

Proceedings of the 17th ACM conference on Computer supported cooperative work & social computing
Special Section on HDR Imaging: Spatially varying image based lighting using HDR-video

Computers and Graphics
Robust Bayesian fitting of 3D morphable model

Proceedings of the 10th European Conference on Visual Media Production
RemoteFusion: real time depth camera fusion for remote collaboration on physical tasks

Proceedings of the 12th ACM SIGGRAPH International Conference on Virtual-Reality Continuum and Its Applications in Industry
A low-latency 3D teleconferencing system with image based approach

Proceedings of the 12th ACM SIGGRAPH International Conference on Virtual-Reality Continuum and Its Applications in Industry
A divide-and-conquer approach to large scene reconstruction with interactive scene analysis and segmentation

Proceedings of the 12th ACM SIGGRAPH International Conference on Virtual-Reality Continuum and Its Applications in Industry
Enabling fine-grained permissions for augmented reality applications with recognizers

SEC'13 Proceedings of the 22nd USENIX conference on Security
A leap-supported, hybrid AR interface approach

Proceedings of the 25th Australian Computer-Human Interaction Conference: Augmentation, Application, Innovation, Collaboration
Restoring surfaces after removing objects in indoor 3D point clouds

Proceedings of the Fourth Symposium on Information and Communication Technology
Time-of-Flight camera based 3D point cloud reconstruction of a car

Computers in Industry
Towards precise real-time 3D difference detection for industrial applications

Computers in Industry
Autonomous navigation and environment modeling for MAVs in 3-D enclosed industrial environments

Computers in Industry
3D normal distributions transform occupancy maps: An efficient representation for mapping in dynamic environments

International Journal of Robotics Research
Pose Depth Volume extraction from RGB-D streams for frontal gait recognition

Journal of Visual Communication and Image Representation
Multi-resolution surfel maps for efficient dense 3D modeling and tracking

Journal of Visual Communication and Image Representation
Live RGB-D camera tracking for television production studios

Journal of Visual Communication and Image Representation
Real-time cloud-based 3D reconstruction and interaction with a stereo smartphone

Proceedings of the 5th ACM Multimedia Systems Conference
Automatic objects segmentation with RGB-D cameras

Journal of Visual Communication and Image Representation
Fast vision-based scene modeling for augmented reality in unprepared man-made environments

Journal of Ambient Intelligence and Smart Environments - Design and Deployment of Intelligent Environments
Personalized animatable avatars from depth data

JVRC '13 Proceedings of the 5th Joint Virtual Reality Conference
SHREC'13 track: retrieval of objects captured with low-cost depth-sensing cameras

3DOR '13 Proceedings of the Sixth Eurographics Workshop on 3D Object Retrieval

Quantified Score

Hi-index	0.02

Visualization

Abstract

We present a system for accurate real-time mapping of complex and arbitrary indoor scenes in variable lighting conditions, using only a moving low-cost depth camera and commodity graphics hardware. We fuse all of the depth data streamed from a Kinect sensor into a single global implicit surface model of the observed scene in real-time. The current sensor pose is simultaneously obtained by tracking the live depth frame relative to the global model using a coarse-to-fine iterative closest point (ICP) algorithm, which uses all of the observed depth data available. We demonstrate the advantages of tracking against the growing full surface model compared with frame-to-frame tracking, obtaining tracking and mapping results in constant time within room sized scenes with limited drift and high accuracy. We also show both qualitative and quantitative results relating to various aspects of our tracking and mapping system. Modelling of natural scenes, in real-time with only commodity sensor and GPU hardware, promises an exciting step forward in augmented reality (AR), in particular, it allows dense surfaces to be reconstructed in real-time, with a level of detail and robustness beyond any solution yet presented using passive computer vision.