Viewing Real-World Faces in 3D

Abstract: We present a data-driven method for estimating the 3D shapes of faces viewed in single, unconstrained photos (aka "in-the-wild"). Our method was designed with an emphasis on robustness and efficiency - with the explicit goal of deployment in real-world applications which reconstruct and display faces in 3D. Our key observation is that for many practical applications, warping the shape of a reference face to match the appearance of a query, is enough to produce realistic impressions of the query's 3D shape. Doing so, however, requires matching visual features between the (possibly very different) query and reference images, while ensuring that a plausible face shape is produced. To this end, we describe an optimization process which seeks to maximize the similarity of appearances and depths, jointly, to those of a reference model. We describe our system for monocular face shape reconstruction and present both qualitative and quantitative experiments, comparing our method against alternative systems, and demonstrating its capabilities. Finally, as a testament to its suitability for real-world applications, we offer an open, on-line implementation of our system, providing unique means of instant 3D viewing of faces appearing in web photos.

Reference:  T. Hassner, Viewing Real-World Faces in 3D, International Conference on Computer Vision (ICCV), Sydney, Austraila, Dec. 2013

Click here for the preprint PDF
Click here for the BibTex

Updates:

• 3-May-2015: Please see followup project on face frontalization for code and data.
• 28-Oct-2013: We've now made the code used for face pose adjustments available online! This includes functions for rendering standard 3D, CG files directly from MATLAB, along with depths and 3D coordinates.

A demo of an early (partial) version of our online system for 3D viewing of faces, presented before Nobel laureate and Israeli President, Mr. Shimon Peres and featured on Channel 10, Israeli TV (in Hebrew).

Last update 3rd of May, 2015