Face Tracking

&

Pose Estimation

Objective:

 

Face Tracking & Pose Estimation project woks on human face tracking and pose estimation in video sequence and related computer vision topics such as the integration with face detector and alignment. The objective of the team is to develop the algorithms and systems which are able to robustly and precisely track the key points on the face and estimate the pose of the head. The technology provides the attention direction of the computer user to help human computer interaction, i.e. User Attention Tracking.

 

The system of "User Attention Tracking in Large Display" automatically tracks a user attention area in a large display or multiple monitors so that the attentive documents or folders in a large display or multiple monitors could be activated. That is, when a user sits in front of a large scale display, the system is able to track the user's head and key facial points in real-time with a single camera, derives the user's attention to different positions within the large screen or among the multiple monitors, and automatically move the cursor to the attention center, activate opened documents or highlight the window within it. In this way, the efficiency of user's interaction in the large display and experience will be greatly improved.

 

Our Solution:

 

Future directions:

 

       

We’re investigating to improve the accuracy and the speed of the face tracking and pose estimation by combining the head pose and the key facial point positions. Our target is to achieve about 5 degree pose estimation precision in yaw and tilt direction in real-time, which enable the user to move his mouse cursor by head instead of hand. Dual camera situation and sophisticated tracking and pose estimation algorithms will also be considered to improve the system performance.

We are also interested with gaze tracking, blink detection, mouth tracking and other face analysis technologies which will contribute to next generation human-computer interface. A new framework of head-driven HCI is under considered. 

Related publication

 

    Y.X. Hu, L.B. Chen, Y. Zhou, H.J. Zhang, "Estimating Face Pose by Facial Asymmetry and Geometry", the 6th Intl. Conf. on Automatic Face and Gesture Recognition (FGR2004)
 

Downloads:

 

Team members & Contacts:

    Past members: