3D Face Reconstruction for Recognition

    

    Overview: According to a generic morphable 3D face model which is learnt off-line based large amount of 3D human face data samples, 3D face reconstruction tries to recover the specific face geometry and texture of a person by single frontal pose, neutral expression and normal illumination face image. After this specific 3D face model is recovered, numerous virtual face samples with different pose, illumination and expressions could be generated to help recognition. 3D face tracking, 3D face animation can also be achieved under the help of this 3D face model.

    Framework:

   

    First, frontal face detection and alignment are utilized to locate a frontal face and the facial feature points within an image, such as the contour points of the face, left and right eyes, mouth and nose. Then, the 3D face shape is reconstructed according to the feature points and a 3D face database. After that, the face model is texture-mapped by projecting the input 2D image onto the 3D face shape. Based on this 3D face model, virtual samples with variant PIE are synthesized to represent the 2D face image space. Finally, face recognition is conducted in this enlarged face subspace after standard normalization of testing sample face images.

   Results: Below are some of our reconstructed/generated face images and the final recognition results comparisons:

   

    Downloads:

  • Demo Video video clips of the screenshot. (Mpeg File, 9.27MB) (The color is not good due to compression, which is much better in our real demo.)
  • Slides Show: 4 PowerPoint slides to animate the framework of our system (PowerPoint File, 9.058M)
  • Sample Images:  include original frontal face images, reconstructed face shapes and reconstructed face models (Sample1.zip 3.227MB, Sample2.zip 1.085MB)

Sample1

 

 

Sample2

    Related publications and works:

  • Y.X. Hu, D.L. Jiang, S.C. Yan, H.J. Zhang, "Automatic 3D Reconstruction for Face Recognition", Accepted by FGR2004 as Oral Presentation
  • Y.X. Hu, S.C. Yan, D.L. Jiang, Y.Zhou, R. Xiao, "Personalized 3D Face Model Reconstruction from Single Image", Demo Proposal Submitted to ECCV2004
  • D. Jiang, Z. Li, Z. Wang, W. Gao, Animating 3D Facial Models with MPEG-4 FaceDefTables, The 35th Annual Simulation Symposium, 2002, pp395-400.
  • D. Jiang, W. Gao, Z. Wang, Z. Li, Animating arbitrary topology 3D facial model using the MPEG-4 FaceDefTalbes, ICMI2002.

·        S. Romdhani, V. Blanz, T. Vetter, “Face Identification by fitting a 3d Morphable Model Using Linear Shape and Texture Error Functions”, In Proc. of European Conf. on Computer Vision, V4, 2002,pp3-19.

·        Face Modeling, MSR, Zhengyou Zhang

   Contacts:

·        Yuxiao HU, (hu3@ifp.uiuc.edu)

·        Lei ZHANG, (leizhang@microsoft.com)

·        Shuicheng YAN, (Math Institute, Peking University)

   Past Members:

  • Lei HUANG(Associate Researcher, Microsoft Research, Asia)
  • Shiguang SHAN (Ph.D Candidate and Assistant Researcher in Institute of Computing Technology, CAS)