Detection and Tracking of Facial Features in Real Time Using a Synergistic Approach of Spatio-Temporal Models and Generalized Hough-Transform Techniques

• Authors: A. Schubert
• Venue: FG '00 Proceedings of the Fourth IEEE International Conference on Automatic Face and Gesture Recognition 2000
• Year: 2000

The proposed algorithm requires the description of the facial features as 3D-polygons (optionally extended by additional intensity information) which are assembled in a 3D-model of the head provided for in separate data files. Detection is achieved by using a special implementation of The Generalized Hough-Transform (GHT) for which the forms are generalized by projecting the 3D-model into the image plane. In the initialization phase a comparatively wide range of relative positions and attitudes between head and camera has to be tested for. Aiming for illumination independence, only information about the sign of the difference between the expected intensities on both sides of the edge of the polygons may be additionally used in GHT. Once a feature is found, further search for the remaining features can be restricted by the use of the 3D-model. The detection of a minimum number of features starts the tracking phase which, is performed by using an Extended Kalman-Filter (EKF) and assuming a first or second order dynamical model for the state variables describing the position and the attitude of the head. Synergistic advantages between GHT and EKF can be realized since the EKF and the projection into the image plane yield a rather good prediction of the forms to be detected by the GHT. This reduces considerably the search space in the image and in the parameter space. On the other hand the GHT offers a solution to the matching problem between image and object features. During the tracking phase monitoring the actual intensities along the edges of the polygons, their assignment to the corresponding 3D-object features, and their use for feature selection during the accumulation process can further enhance the GHT. The algorithm runs on a Dual Pentium II 333 MHz with a cycle time of 40ms in real time.