3D Face Pose Detection and Tracking

                    Homepage                    Research                    Patents                    Academic                   Activities                    Contact me

"Real-Time Robust 3D Face Pose Detection And Tracking"

•       Tutor: Professor Xiong Rong, Doc. Wang Junan
• Lab. of Robotics for Home Service, Zhejiang University
• currently undergoing, starting from June, 2011

   Introduction:                                                                      

      This project is undertaken in the Lab.of Robotics for Home Service, Zhejiang University, the team of which participates in the Robocup Competition annually. Vision for home  service robots is of crucial importance. Only by relying on the accurate recognition of human and objects can the robots really perform their tasks such as assisting the blind, cooking meals. The goal for the project I'm in is maily about precise recognition of human face of different poses, including turning and tilting. By combining such information with unique texture of each individual face, the robot can complete the task of human identification. What we are trying to achieve is for the robots to be able to recognize their acquaintances, meaning that by telling the robot who you are the first time, the robot can identify you again in a different surrounding, even with the presence of strangers. Both the intrinsic (shape...) and extrinsic (illumination...) characters of human face varies significantly, hence robust recognition is the foundational prerequisite. By being able to detecting human faces of different poses, we can follow the movement of a person more exactly, therefore, reconstructing the image that is most suitable for recognition.

   Project Procedure                                                            

    This project develops algorithm for fast human face pose detection and tracking based on following processes:

1. Training of 3D human-face database:  manually sellecte 3D feature points and  normalize  image data onto the same coordinate, then use PCA to abtain eigenvectors that best respresent each training face. The mean face of the training set can be represented as X=+Qb, where Q is the eigenvector matrix and b represent an estimate of propability density of face space.
2. Using ASM method for face detection and tracking: build the ASM model for 3D human face, minimize the distance between the input face with the model using following equation: min||X_input-A[R,T](+Qb)||, X_input represents the targeted input face, R and T represents the rotation and spatial shift matrix of the image, A represents the calibration vector of the camera, related to T.
3. Using iterative searching method to find R and T to minimize the distance, hence get the optimun rotation and texture vector for targeted face.

      Innovations and Difficulties:                                          

1. Take advantage of the symmetry of human face and select fewer feature points to speed up the process
2. Use 3D ASM model rather than 3D morphable model so that less information has to be caltulated
3. Since the trainig set only has frontal face, the outline of profile face is sometimes inaccurate.

      My Role in The Project:                                                 

   I take on this project from scratch from developing refined algorithm to actual programming in C++, this project is currently undergoing...