A survey of approaches and challenges in 3D and multi-modal 3D+2D face recognition

Authors:
Kevin W. Bowyer;Kyong Chang;Patrick Flynn
Affiliations:
Department of Computer Science and Engineering, University of Notre Dame, Notre Dame, IN 46556, USA;Department of Computer Science and Engineering, University of Notre Dame, Notre Dame, IN 46556, USA;Department of Computer Science and Engineering, University of Notre Dame, Notre Dame, IN 46556, USA
Venue:
Computer Vision and Image Understanding
Year:
2006

Citing 32
Cited 117

The FERET Evaluation Methodology for Face-Recognition Algorithms

IEEE Transactions on Pattern Analysis and Machine Intelligence
Face verification from 3D and grey level clues

Pattern Recognition Letters
Use of depth and colour eigenfaces for face recognition

Pattern Recognition Letters
Curvature-Based Face Surface Recognition Using Spherical Correlation - Principal Directions for Curved Object Recognition

FG '98 Proceedings of the 3rd. International Conference on Face & Gesture Recognition
3D Human Face Recognition Using Point Signature

FG '00 Proceedings of the Fourth IEEE International Conference on Automatic Face and Gesture Recognition 2000
Face Recognition Using Range Images

VSMM '97 Proceedings of the 1997 International Conference on Virtual Systems and MultiMedia
Face Recognition Based on Fitting a 3D Morphable Model

IEEE Transactions on Pattern Analysis and Machine Intelligence
Comparison and Combination of Ear and Face Images in Appearance-Based Biometrics

IEEE Transactions on Pattern Analysis and Machine Intelligence
Face Modeling and Recognition in 3-D

AMFG '03 Proceedings of the IEEE International Workshop on Analysis and Modeling of Faces and Gestures
Face recognition: A literature survey

ACM Computing Surveys (CSUR)
Deformation Analysis for 3D Face Matching

WACV-MOTION '05 Proceedings of the Seventh IEEE Workshops on Application of Computer Vision (WACV/MOTION'05) - Volume 1 - Volume 01
Integrating Range and Texture Information for 3D Face Recognition

WACV-MOTION '05 Proceedings of the Seventh IEEE Workshops on Application of Computer Vision (WACV/MOTION'05) - Volume 1 - Volume 01
An Evaluation of Multimodal 2D+3D Face Biometrics

IEEE Transactions on Pattern Analysis and Machine Intelligence
Three-Dimensional Face Recognition

International Journal of Computer Vision
Accuracy of 3D Scanning Technologies in a Face Scanning Scenario

3DIM '05 Proceedings of the Fifth International Conference on 3-D Digital Imaging and Modeling
Overview of the Face Recognition Grand Challenge

CVPR '05 Proceedings of the 2005 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'05) - Volume 1 - Volume 01
Empirical Evaluation of Advanced Ear Biometrics

CVPR '05 Proceedings of the 2005 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'05) - Workshops - Volume 03
Performance of Geometrix ActiveID^TM 3D Face Recognition Engine on the FRGC Data

CVPR '05 Proceedings of the 2005 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'05) - Workshops - Volume 03
Adaptive Rigid Multi-region Selection for Handling Expression Variation in 3D Face Recognition

CVPR '05 Proceedings of the 2005 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'05) - Workshops - Volume 03
A Prescreener for 3D Face Recognition Using Radial Symmetry and the Hausdorff Fraction

CVPR '05 Proceedings of the 2005 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'05) - Workshops - Volume 03
3D Face Recognition using Mapped Depth Images

CVPR '05 Proceedings of the 2005 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'05) - Workshops - Volume 03
Evaluation of 3D Face Recognition in the presence of facial expressions: an Annotated Deformable Model approach

CVPR '05 Proceedings of the 2005 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'05) - Workshops - Volume 03
Strategies and Benefits of Fusion of 2D and 3D Face Recognition

CVPR '05 Proceedings of the 2005 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'05) - Workshops - Volume 03
A Fast Algorithm for ICP-Based 3D Shape Biometrics

AUTOID '05 Proceedings of the Fourth IEEE Workshop on Automatic Identification Advanced Technologies
Expression-invariant 3D face recognition

AVBPA'03 Proceedings of the 4th international conference on Audio- and video-based biometric person authentication
Integration of 2D and 3D images for enhanced face authentication

FGR' 04 Proceedings of the Sixth IEEE international conference on Automatic face and gesture recognition
Automatic 3D face recognition combining global geometric features with local shape variation information

FGR' 04 Proceedings of the Sixth IEEE international conference on Automatic face and gesture recognition
Evaluation of automatic 4D face recognition using surface and texture registration

FGR' 04 Proceedings of the Sixth IEEE international conference on Automatic face and gesture recognition
A survey of 3d face recognition methods

AVBPA'05 Proceedings of the 5th international conference on Audio- and Video-Based Biometric Person Authentication
Local feature based 3d face recognition

AVBPA'05 Proceedings of the 5th international conference on Audio- and Video-Based Biometric Person Authentication
Rank-Based decision fusion for 3d shape-based face recognition

AVBPA'05 Proceedings of the 5th international conference on Audio- and Video-Based Biometric Person Authentication
Face localization and authentication using color and depth images

IEEE Transactions on Image Processing

HCI Beyond the GUI: Design for Haptic, Speech, Olfactory, and Other Nontraditional Interfaces

HCI Beyond the GUI: Design for Haptic, Speech, Olfactory, and Other Nontraditional Interfaces
Image Level Fusion Method for Multimodal 2D + 3D Face Recognition

ICIAR '08 Proceedings of the 5th international conference on Image Analysis and Recognition
Intelligent Assistive Exoskeleton with Vision Based Interface

ICOST '08 Proceedings of the 6th international conference on Smart Homes and Health Telematics
Automatic Feature Extraction from 3D Range Images of Skulls

IWCF '08 Proceedings of the 2nd international workshop on Computational Forensics
3D Face Recognition by Local Shape Difference Boosting

ECCV '08 Proceedings of the 10th European Conference on Computer Vision: Part I
3D Face Recognition Benchmarks on the Bosphorus Database with Focus on Facial Expressions

Biometrics and Identity Management
Comparison of Visible, Thermal Infra-Red and Range Images for Face Recognition

PSIVT '09 Proceedings of the 3rd Pacific Rim Symposium on Advances in Image and Video Technology
A Method for Robust Nose Tip Location across Pose Variety in 3D Face Data

CAR '09 Proceedings of the 2009 International Asia Conference on Informatics in Control, Automation and Robotics
3D Face Recognition Using Joint Differential Invariants

ICB '09 Proceedings of the Third International Conference on Advances in Biometrics
Nasal Region-Based 3D Face Recognition under Pose and Expression Variations

ICB '09 Proceedings of the Third International Conference on Advances in Biometrics
Faceprint: Fusion of Local Features for 3D Face Recognition

ICB '09 Proceedings of the Third International Conference on Advances in Biometrics
Facial Reconstruction and Alignment Using Photometric Stereo and Surface Fitting

IbPRIA '09 Proceedings of the 4th Iberian Conference on Pattern Recognition and Image Analysis
Score Level Fusion of Ear and Face Local 3D Features for Fast and Expression-Invariant Human Recognition

ICIAR '09 Proceedings of the 6th International Conference on Image Analysis and Recognition
Landmark Localisation in 3D Face Data

AVSS '09 Proceedings of the 2009 Sixth IEEE International Conference on Advanced Video and Signal Based Surveillance
A Critical Assessment of 2D and 3D Face Recognition Algorithms

AVSS '09 Proceedings of the 2009 Sixth IEEE International Conference on Advanced Video and Signal Based Surveillance
Three-dimensional facial surface modeling applied to recognition

Engineering Applications of Artificial Intelligence
3D face recognition using depth-based features

VIIP '07 The Seventh IASTED International Conference on Visualization, Imaging and Image Processing
3D face recognition with sparse spherical representations

Pattern Recognition
A survey of the effects of aging on biometric identity verification

International Journal of Biometrics
Binary neural network based 3D facial feature localization

IJCNN'09 Proceedings of the 2009 international joint conference on Neural Networks
Uncorrelated multilinear principal component analysis for unsupervised multilinear subspace learning

IEEE Transactions on Neural Networks
A semi-supervised support vector machine based algorithm for face recognition

SMC'09 Proceedings of the 2009 IEEE international conference on Systems, Man and Cybernetics
A coarse-to-fine curvature analysis-based rotation invariant 3D face landmarking

BTAS'09 Proceedings of the 3rd IEEE international conference on Biometrics: Theory, applications and systems
A biometric database with rotating head videos and hand- drawn face sketches

BTAS'09 Proceedings of the 3rd IEEE international conference on Biometrics: Theory, applications and systems
Point-pair descriptors for 3D facial landmark localisation

BTAS'09 Proceedings of the 3rd IEEE international conference on Biometrics: Theory, applications and systems
Local feature hashing for face recognition

BTAS'09 Proceedings of the 3rd IEEE international conference on Biometrics: Theory, applications and systems
Towards 3D-aided profile-based face recognition

BTAS'09 Proceedings of the 3rd IEEE international conference on Biometrics: Theory, applications and systems
Partial matching of interpose 3D facial data for face recognition

BTAS'09 Proceedings of the 3rd IEEE international conference on Biometrics: Theory, applications and systems
Human-robot interfaces for social interaction

International Journal of Robotics and Automation
View-based eigenspaces with mixture of experts for view-independent face recognition

MCS'07 Proceedings of the 7th international conference on Multiple classifier systems
Face recognition based on 2D and 3D features

KES'07/WIRN'07 Proceedings of the 11th international conference, KES 2007 and XVII Italian workshop on neural networks conference on Knowledge-based intelligent information and engineering systems: Part I
Towards pose-invariant 2D face classification for surveillance

AMFG'07 Proceedings of the 3rd international conference on Analysis and modeling of faces and gestures
Discriminating 3D faces by statistics of depth differences

ACCV'07 Proceedings of the 8th Asian conference on Computer vision - Volume Part II
Experimental comparison among 3D innovative face recognition frameworks

ICIC'09 Proceedings of the Intelligent computing 5th international conference on Emerging intelligent computing technology and applications
Robust face recognition based on illumination invariant in nonsubsampled contourlet transform domain

Neurocomputing
Scaling iterative closest point algorithm for registration of m-D point sets

Journal of Visual Communication and Image Representation
3D facial visualization through adaptive spread spectrum synchronous scalable (A4S) data hiding

ICIP'09 Proceedings of the 16th IEEE international conference on Image processing
Asymmetric 3D/2D face recognition based on LBP facial representation and canonical correlation analysis

ICIP'09 Proceedings of the 16th IEEE international conference on Image processing
3D-aided profile-based face recognition

ICIP'09 Proceedings of the 16th IEEE international conference on Image processing
Introduction to the special issue on recent advances in biometrics

IEEE Transactions on Systems, Man, and Cybernetics, Part A: Systems and Humans - Special issue on recent advances in biometrics
Assessing the uniqueness and permanence of facial actions for use in biometric applications

IEEE Transactions on Systems, Man, and Cybernetics, Part A: Systems and Humans - Special issue on recent advances in biometrics
Tracking vertex flow and model adaptation for three-dimensional spatiotemporal face analysis

IEEE Transactions on Systems, Man, and Cybernetics, Part A: Systems and Humans - Special issue on recent advances in biometrics
Hand-drawn face sketch recognition by humans and a PCA-based algorithm for forensic applications

IEEE Transactions on Systems, Man, and Cybernetics, Part A: Systems and Humans - Special issue on recent advances in biometrics
Toward unconstrained ear recognition from two-dimensional images

IEEE Transactions on Systems, Man, and Cybernetics, Part A: Systems and Humans - Special issue on recent advances in biometrics
From 3D Point Clouds to Pose-Normalised Depth Maps

International Journal of Computer Vision
On the Repeatability and Quality of Keypoints for Local Feature-based 3D Object Retrieval from Cluttered Scenes

International Journal of Computer Vision
Subspace methods for retrieval of general 3D models

Computer Vision and Image Understanding
3D face reconstructions from photometric stereo using near infrared and visible light

Computer Vision and Image Understanding
Regional registration for expression resistant 3-D face recognition

IEEE Transactions on Information Forensics and Security
Multimedia sensor fusion for retrieving identity in biometric access control systems

ACM Transactions on Multimedia Computing, Communications, and Applications (TOMCCAP)
Recognition of 3D faces with missing parts based on profile networks

Proceedings of the ACM workshop on 3D object retrieval
Human computing and machine understanding of human behavior: a survey

ICMI'06/IJCAI'07 Proceedings of the ICMI 2006 and IJCAI 2007 international conference on Artifical intelligence for human computing
High-resolution object deformation reconstruction with active range camera

Proceedings of the 32nd DAGM conference on Pattern recognition
3D deformable face tracking with a commodity depth camera

ECCV'10 Proceedings of the 11th European conference on computer vision conference on Computer vision: Part III
Multiclient identification system using adaptive probabilistic model

EURASIP Journal on Advances in Signal Processing - Special issue on advanced image processing for defense and security applications
Online learning from local features for video-based face recognition

Pattern Recognition
Identity retrieval in biometric access control systems using multimedia fusion

ICONIP'10 Proceedings of the 17th international conference on Neural information processing: models and applications - Volume Part II
3D face recognition based on local shape patterns and sparse representation classifier

MMM'11 Proceedings of the 17th international conference on Advances in multimedia modeling - Volume Part I
A survey of multilinear subspace learning for tensor data

Pattern Recognition
PCA-based image recombination for multimodal 2D+3D face recognition

Image and Vision Computing
Three-Dimensional Occlusion Detection and Restoration of Partially Occluded Faces

Journal of Mathematical Imaging and Vision
Fast and Accurate 3D Face Recognition

International Journal of Computer Vision
Forensic identification by computer-aided craniofacial superimposition: A survey

ACM Computing Surveys (CSUR)
TOPLEAGUE and BUNDESLIGA MANAGER: new generation online soccer games

RoboCup 2010
2.5D Elastic graph matching

Computer Vision and Image Understanding
Mixed 2D-3D information for face recognition

Transactions on edutainment V
Real-time 3D face recognition with the integration of depth and intensity images

ICIAR'11 Proceedings of the 8th international conference on Image analysis and recognition - Volume Part II
Profile-based 3D-aided face recognition

Pattern Recognition
Efficient Detection and Recognition of 3D Ears

International Journal of Computer Vision
3D face recognition based on sparse representation

The Journal of Supercomputing
Hybrid face recognition based on real-time multi-camera stereo-matching

ISVC'11 Proceedings of the 7th international conference on Advances in visual computing - Volume Part II
An improvement to matrix-based LDA

AICI'11 Proceedings of the Third international conference on Artificial intelligence and computational intelligence - Volume Part III
Face detection methods

ACS'11 Proceedings of the 11th WSEAS international conference on Applied computer science
3D partial face matching using local shape descriptors

J-HGBU '11 Proceedings of the 2011 joint ACM workshop on Human gesture and behavior understanding
3D face recognition based on non-iterative registration and single b-spline patch modelling techniques

ACIVS'06 Proceedings of the 8th international conference on Advanced Concepts For Intelligent Vision Systems
Ultra fast GPU assisted face recognition based on 3d geometry and texture data

ICIAR'06 Proceedings of the Third international conference on Image Analysis and Recognition - Volume Part II
Face recognition using 2d and 3d multimodal local features

ISVC'06 Proceedings of the Second international conference on Advances in Visual Computing - Volume Part I
2D representation of facial surfaces for multi-pose 3D face recognition

Pattern Recognition Letters
Robust 3d face recognition based on rejection and adaptive region selection

ACCV'09 Proceedings of the 9th Asian conference on Computer Vision - Volume Part III
Distinguishing Facial Features for Ethnicity-Based 3D Face Recognition

ACM Transactions on Intelligent Systems and Technology (TIST)
Spatial feature interdependence matrix (SFIM): a robust descriptor for face recognition

PSIVT'11 Proceedings of the 5th Pacific Rim conference on Advances in Image and Video Technology - Volume Part I
A review of recent advances in 3D ear- and expression-invariant face biometrics

ACM Computing Surveys (CSUR)
Face recognition in 2D and 2.5D using ridgelets and photometric stereo

Pattern Recognition
Two dimensional synthetic face generation and verification using set estimation technique

Computer Vision and Image Understanding
On the simultaneous recognition of identity and expression from BU-3DFE datasets

Pattern Recognition Letters
3D face recognition using an expression insensitive dynamic mask

ICISP'12 Proceedings of the 5th international conference on Image and Signal Processing
Robust sparse bounding sphere for 3D face recognition

Image and Vision Computing
An efficient 3D face recognition approach based on the fusion of novel local low-level features

Pattern Recognition
Efficient 3D face recognition handling facial expression and hair occlusion

Image and Vision Computing
Hi4D-ADSIP 3-D dynamic facial articulation database

Image and Vision Computing
Reshaping 3D facial scans for facial appearance modeling and 3D facial expression analysis

Image and Vision Computing
Face recognition by SVMs classification and manifold learning of 2D and 3D radial geodesic distances

EG 3DOR'08 Proceedings of the 1st Eurographics conference on 3D Object Retrieval
Partial match of 3D faces using facial curves between SIFT keypoints

EG 3DOR'11 Proceedings of the 4th Eurographics conference on 3D Object Retrieval
Method to evaluate pose variability in automatic face recognition performance

International Journal of Biometrics
Automatic facial pose determination of 3D range data for face model and expression identification

ICB'07 Proceedings of the 2007 international conference on Advances in Biometrics
Keypoint identification and feature-based 3D face recognition

ICB'07 Proceedings of the 2007 international conference on Advances in Biometrics
Robust 3D face recognition from expression categorisation

ICB'07 Proceedings of the 2007 international conference on Advances in Biometrics
GPU accelerated 3D face registration / recognition

ICB'07 Proceedings of the 2007 international conference on Advances in Biometrics
Illumination normalization using self-lighting ratios for 3d2d face recognition

ECCV'12 Proceedings of the 12th international conference on Computer Vision - Volume 2
Robust learning from normals for 3d face recognition

ECCV'12 Proceedings of the 12th international conference on Computer Vision - Volume 2
meshSIFT: Local surface features for 3D face recognition under expression variations and partial data

Computer Vision and Image Understanding
A structured template based 3D face recognition approach

Proceedings of the 27th Conference on Image and Vision Computing New Zealand
Multi-pose 3D face recognition based on 2D sparse representation

Journal of Visual Communication and Image Representation
Cortical 3D Face and Object Recognition Using 2D Projections

International Journal of Creative Interfaces and Computer Graphics
3D aided face recognition across pose variations

CCBR'12 Proceedings of the 7th Chinese conference on Biometric Recognition
3D facial landmark localization via a local surface descriptor HoSNI

IScIDE'12 Proceedings of the third Sino-foreign-interchange conference on Intelligent Science and Intelligent Data Engineering
A unified probabilistic framework for automatic 3D facial expression analysis based on a Bayesian belief inference and statistical feature models

Image and Vision Computing
3D face recognition using local binary patterns

Signal Processing
Special Section on 3D Object Retrieval: Matching 3D face scans using interest points and local histogram descriptors

Computers and Graphics
Pose invariant face recognition using biological inspired features based on ensemble of classifiers

Journal of Visual Communication and Image Representation
Letters: Fuzzy MSD based feature extraction method for face recognition

Neurocomputing
An efficient 3D face recognition approach using local geometrical signatures

Pattern Recognition
Robust 3D face capture using example-based photometric stereo

Computers in Industry
A new framework for 3D face reconstruction for self-occluded images

International Journal of Computational Vision and Robotics
A novel framework for automatic 3D face recognition using quality assessment

Multimedia Tools and Applications
Expression-robust 3D face recognition via weighted sparse representation of multi-scale and multi-component local normal patterns

Neurocomputing
Geometric histograms of 3D keypoints for face identification with missing parts

3DOR '13 Proceedings of the Sixth Eurographics Workshop on 3D Object Retrieval

Quantified Score

Hi-index	0.00

Visualization

Abstract

This survey focuses on recognition performed by matching models of the three-dimensional shape of the face, either alone or in combination with matching corresponding two-dimensional intensity images. Research trends to date are summarized, and challenges confronting the development of more accurate three-dimensional face recognition are identified. These challenges include the need for better sensors, improved recognition algorithms, and more rigorous experimental methodology.