Accurate 3D Face Reconstruction with Weakly-Supervised Learning: From Single Image to Image Set
Accurate 3D Face Reconstruction with Weakly-Supervised Learning: From Single Image to Image Set
Yu Deng*1,2 Jiaolong Yang2 Sicheng Xu3,2 Dong Chen2 Yunde Jia3 Xin Tong2

Abstract
Recently, deep learning based 3D face reconstruction methods have shown promising results in both quality and efficiency.However, training deep neural networks typically requires a large volume of data, whereas face images with ground-truth 3D face shapes are scarce. In this paper, we propose a novel deep 3D face reconstruction approach that 1) leverages a robust, hybrid loss function for weakly-supervised learning which takes into account both low-level and perception-level information for supervision, and 2) performs multi-image face reconstruction by exploiting complementary information from different images for shape aggregation. Our method is fast, accurate, and robust to occlusion and large pose. We provide comprehensive experiments on three datasets, systematically comparing our method with fifteen recent methods and demonstrating its state-of-the-art performance.
Code Repositories
Benchmarks
| Benchmark | Methodology | Metrics |
|---|---|---|
| 3d-face-reconstruction-on-florence | Deep3DFaceReconstruction | RMSE Cooperative: 1.66 RMSE Indoor: 1.66 RMSE Outdoor: 1.69 |
| 3d-face-reconstruction-on-florence | Deng | RMSE Cooperative: 1.60 RMSE Indoor: 1.61 |
| 3d-face-reconstruction-on-now-benchmark-1 | Deng et al. 2019 | Mean Reconstruction Error (mm): 1.54 Median Reconstruction Error: 1.23 Stdev Reconstruction Error (mm): 1.29 |
| 3d-face-reconstruction-on-now-benchmark-1 | Deep3DFaceRecon PyTorch | Mean Reconstruction Error (mm): 1.41 Median Reconstruction Error: 1.11 Stdev Reconstruction Error (mm): 1.21 |
| 3d-face-reconstruction-on-realy | Deep3D | @cheek: 1.528 (±0.501) @forehead: 2.015 (±0.449) @mouth: 1.368 (±0.439) @nose: 1.719 (±0.354) all: 1.657 |
| 3d-face-reconstruction-on-realy-side-view | Deep3D | @cheek: 1.528 (±0.517) @forehead: 2.074 (±0.486) @mouth: 1.411 (±0.395) @nose: 1.749 (±0.343) all: 1.657 |
Build AI with AI
From idea to launch — accelerate your AI development with free AI co-coding, out-of-the-box environment and best price of GPUs.