Stable Real-Time 3D Tracking Using Online and Offline Information

Authors:
Luca Vacchetti;Vincent Lepetit;Pascal Fua
Affiliations:
-;-;IEEE
Venue:
IEEE Transactions on Pattern Analysis and Machine Intelligence
Year:
2004

Citing 18
Cited 67

Robust model-based motion tracking through the integration of search and estimation

International Journal of Computer Vision
Three-dimensional computer vision: a geometric viewpoint

Three-dimensional computer vision: a geometric viewpoint
Sequential Updating of Projective and Affine Structure from Motion

International Journal of Computer Vision
Efficient Region Tracking With Parametric Models of Geometry and Illumination

IEEE Transactions on Pattern Analysis and Machine Intelligence
Computer vision methods for registration: mixing 3D knowledge and 2D correspondences for accurate image composition

IWAR '98 Proceedings of the international workshop on Augmented reality : placing artificial objects in real scenes: placing artificial objects in real scenes
Fast, Reliable Head Tracking under Varying Illumination: An Approach Based on Registration of Texture-Mapped 3D Models

IEEE Transactions on Pattern Analysis and Machine Intelligence
Multiple view geometry in computer visiond

Multiple view geometry in computer visiond
Recursive Estimation of Motion, Structure, and Focal Length

IEEE Transactions on Pattern Analysis and Machine Intelligence
Real-Time Visual Tracking of Complex Structures

IEEE Transactions on Pattern Analysis and Machine Intelligence
Hyperplane Approximation for Template Matching

IEEE Transactions on Pattern Analysis and Machine Intelligence
Automatic Camera Recovery for Closed or Open Image Sequences

ECCV '98 Proceedings of the 5th European Conference on Computer Vision-Volume I - Volume I
The Integration of Optical Flow and Deformable Models with Applications to Human Face Shape and Motion Estimation

CVPR '96 Proceedings of the 1996 Conference on Computer Vision and Pattern Recognition (CVPR '96)
Adaptive tracking and model registration across distinct aspects

IROS '95 Proceedings of the International Conference on Intelligent Robots and Systems-Volume 1 - Volume 1
Online 6 DOF Augmented Reality Registration from Natural Features

ISMAR '02 Proceedings of the 1st International Symposium on Mixed and Augmented Reality
Self-Calibration and Metric Reconstruction in spite of Varying and Unknown Internal Camera Parameters

ICCV '98 Proceedings of the Sixth International Conference on Computer Vision
Fully Automated and Stable Registration for Augmented Reality Applications

ISMAR '03 Proceedings of the 2nd IEEE/ACM International Symposium on Mixed and Augmented Reality
Distinctive Image Features from Scale-Invariant Keypoints

International Journal of Computer Vision
Natural feature tracking for augmented reality

IEEE Transactions on Multimedia

Automated Initialization for Marker-Less Tracking: A Sensor Fusion Approach

ISMAR '04 Proceedings of the 3rd IEEE/ACM International Symposium on Mixed and Augmented Reality
Combining head tracking and mouse input for a GUI on multiple monitors

CHI '05 Extended Abstracts on Human Factors in Computing Systems
A Balanced Approach to 3D Tracking from Image Streams

ISMAR '05 Proceedings of the 4th IEEE/ACM International Symposium on Mixed and Augmented Reality
Monocular model-based 3D tracking of rigid objects

Foundations and Trends® in Computer Graphics and Vision
A Framework for Model-Based Tracking Experiments in Image Sequences

International Journal of Computer Vision
Three-Dimensional Shape Knowledge for Joint Image Segmentation and Pose Tracking

International Journal of Computer Vision
Model-Based Tracking by Classification in a Tiny Discrete Pose Space

IEEE Transactions on Pattern Analysis and Machine Intelligence
Real-time Hybrid Tracking using Edge and Texture Information

International Journal of Robotics Research
Robust real-time vision for a personal service robot

Computer Vision and Image Understanding
Mutual Information-Based 3D Object Tracking

International Journal of Computer Vision
Pose Robust Face Tracking by Combining Active Appearance Models and Cylinder Head Models

International Journal of Computer Vision
Markerless Vision-Based Tracking of Partially Known 3D Scenes for Outdoor Augmented Reality Applications

ISVC '08 Proceedings of the 4th International Symposium on Advances in Visual Computing
A mapping and localization framework for scalable appearance-based navigation

Computer Vision and Image Understanding
Online camera pose estimation in partially known and dynamic scenes

ISMAR '06 Proceedings of the 5th IEEE and ACM International Symposium on Mixed and Augmented Reality
A mobile markerless AR system for maintenance and repair

ISMAR '06 Proceedings of the 5th IEEE and ACM International Symposium on Mixed and Augmented Reality
An Industrial Augmented Reality Solution For Discrepancy Check

ISMAR '07 Proceedings of the 2007 6th IEEE and ACM International Symposium on Mixed and Augmented Reality
Marker Less Vision-Based Tracking of Partially Known 3D Scenes for Outdoor Augmented Reality Applications

ICCVG 2008 Proceedings of the International Conference on Computer Vision and Graphics: Revised Papers
How to augment the second image? Recovery of the translation scale in image to image registration

ISMAR '08 Proceedings of the 7th IEEE/ACM International Symposium on Mixed and Augmented Reality
Multiple 3D Object tracking for augmented reality

ISMAR '08 Proceedings of the 7th IEEE/ACM International Symposium on Mixed and Augmented Reality
Marker-Less Tracking for Multi-layer Authoring in AR Books

ICEC '09 Proceedings of the 8th International Conference on Entertainment Computing
Robust object tracking with a case-base updating strategy

IJCAI'07 Proceedings of the 20th international joint conference on Artifical intelligence
Vision for Robotics

Foundations and Trends in Robotics
Complementing visual tracking of moving targets by fusion of tactile sensing

Robotics and Autonomous Systems
Standalone edge-based markerless tracking of fully 3-dimensional objects for handheld augmented reality

Proceedings of the 16th ACM Symposium on Virtual Reality Software and Technology
Identifying noncooperative subjects at a distance using face images and inferred three-dimensional face models

IEEE Transactions on Systems, Man, and Cybernetics, Part A: Systems and Humans - Special section: Best papers from the 2007 biometrics: Theory, applications, and systems (BTAS 07) conference
HERB: a home exploring robotic butler

Autonomous Robots
Landmark Real-Time Recognition and Positioning for Pedestrian Navigation

CIARP '09 Proceedings of the 14th Iberoamerican Conference on Pattern Recognition: Progress in Pattern Recognition, Image Analysis, Computer Vision, and Applications
Structure and appearance features for robust 3D facial actions tracking

ICME'09 Proceedings of the 2009 IEEE international conference on Multimedia and Expo
Articulated object tracking by rendering consistent appearance parts

ICRA'09 Proceedings of the 2009 IEEE international conference on Robotics and Automation
Object recognition and full pose registration from a single image for robotic manipulation

ICRA'09 Proceedings of the 2009 IEEE international conference on Robotics and Automation
Real-time face tracking and pose estimation with partitioned sampling and relevance vector machine

ICRA'09 Proceedings of the 2009 IEEE international conference on Robotics and Automation
Object recognition and pose estimation using color cooccurrence histograms and geometric modeling

Image and Vision Computing
Registration based on scene recognition and natural features tracking techniques for wide-area augmented reality systems

IEEE Transactions on Multimedia
Interoperable RT component for object detection and 3D pose estimation for service robots

IROS'09 Proceedings of the 2009 IEEE/RSJ international conference on Intelligent robots and systems
Person-independent monocular tracking of face and facial actions with multilinear models

AMFG'07 Proceedings of the 3rd international conference on Analysis and modeling of faces and gestures
Integrating multiple visual cues for robust real-time 3D face tracking

AMFG'07 Proceedings of the 3rd international conference on Analysis and modeling of faces and gestures
Model-assisted 3D face reconstruction from video

AMFG'07 Proceedings of the 3rd international conference on Analysis and modeling of faces and gestures
Real-time 3D face tracking with mutual information and active contours

ISVC'07 Proceedings of the 3rd international conference on Advances in visual computing - Volume Part I
Drift-correcting template update strategy for precision feature point tracking

Image and Vision Computing
From rendering to tracking point-based 3D models

Image and Vision Computing
AffTrack: robust tracking of features in variable-zoom videos

ICIP'09 Proceedings of the 16th IEEE international conference on Image processing
A graphical model based solution to the facial feature point tracking problem

Image and Vision Computing
Robust: real-time 3D face tracking from a monocular view

Journal on Image and Video Processing
Mobile Augmented Reality: Real-time and accurate extrinsic camera parameter estimation using feature landmark database for augmented reality

Computers and Graphics
3D real-time positioning for autonomous navigation using a nine-point landmark

Pattern Recognition
Probabilistic object models for pose estimation in 2D images

DAGM'11 Proceedings of the 33rd international conference on Pattern recognition
Markerless augmented reality using a robust point transferring method

MMM'07 Proceedings of the 13th International conference on Multimedia Modeling - Volume Part II
Tracking planes with large interframe displacement by fusing template and point based approaches

Proceedings of the 10th International Conference on Virtual Reality Continuum and Its Applications in Industry
A mixed reality system for virtual glasses try-on

Proceedings of the 10th International Conference on Virtual Reality Continuum and Its Applications in Industry
A practical guide to marker based and hybrid visual registration for AR industrial applications

CAIP'05 Proceedings of the 11th international conference on Computer Analysis of Images and Patterns
Real-time non-rigid shape recovery via active appearance models for augmented reality

ECCV'06 Proceedings of the 9th European conference on Computer Vision - Volume Part I
Vision-based augmented reality visual guidance with keyframes

CGI'06 Proceedings of the 24th international conference on Advances in Computer Graphics
An investigation of model bias in 3d face tracking

AMFG'05 Proceedings of the Second international conference on Analysis and Modelling of Faces and Gestures
Automatic 3d facial expression analysis in videos

AMFG'05 Proceedings of the Second international conference on Analysis and Modelling of Faces and Gestures
Real-time modeling of face deformation for 3d head pose estimation

AMFG'05 Proceedings of the Second international conference on Analysis and Modelling of Faces and Gestures
Fusion of 3d and appearance models for fast object detection and pose estimation

ACCV'06 Proceedings of the 7th Asian conference on Computer Vision - Volume Part II
Integation methods of model-free features for 3d tracking

SCIA'05 Proceedings of the 14th Scandinavian conference on Image Analysis
High accuracy optical flow serves 3-d pose tracking: exploiting contour and flow based constraints

ECCV'06 Proceedings of the 9th European conference on Computer Vision - Volume Part II
Robust pose estimation with 3d textured models

PSIVT'06 Proceedings of the First Pacific Rim conference on Advances in Image and Video Technology
Fusing template and point information to track planes with large interframe displacement

MMM'12 Proceedings of the 18th international conference on Advances in Multimedia Modeling
Combined online and offline information for tracking facial feature points

ICIRA'12 Proceedings of the 5th international conference on Intelligent Robotics and Applications - Volume Part I
iAR: an exploratory augmented reality system for mobile devices

Proceedings of the 18th ACM symposium on Virtual reality software and technology
Robust 2D-3D matching for 3D object pose detection

SIGGRAPH Asia 2012 Technical Briefs
Markerless tracking algorithm based on 3d model for augmented reality system

IScIDE'12 Proceedings of the third Sino-foreign-interchange conference on Intelligent Science and Intelligent Data Engineering
Model-based pose estimation for rigid objects

ICVS'13 Proceedings of the 9th international conference on Computer Vision Systems
Robust multi-hypothesis 3D object pose tracking

ICVS'13 Proceedings of the 9th international conference on Computer Vision Systems
Tracking-by-detection of multiple persons by a resample-move particle filter

Machine Vision and Applications

Quantified Score

Hi-index	0.14

Visualization

Abstract

We propose an efficient real-time solution for tracking rigid objects in 3D using a single camera that can handle large camera displacements, drastic aspect changes, and partial occlusions. While commercial products are already available for offline camera registration, robust online tracking remains an open issue because many real-time algorithms described in the literature still lack robustness and are prone to drift and jitter. To address these problems, we have formulated the tracking problem in terms of local bundle adjustment and have developed a method for establishing image correspondences that can equally well handle short and wide-baseline matching. We then can merge the information from preceding frames with that provided by a very limited number of keyframes created during a training stage, which results in a real-time tracker that does not jitter or drift and can deal with significant aspect changes.