Jianzhu Guo Xiangyu Zhu Yang Yang Fan Yang Zhen Lei Stan Z. Li

Abstract
Existing methods of 3D dense face alignment mainly concentrate on accuracy,thus limiting the scope of their practical applications. In this paper, wepropose a novel regression framework named 3DDFA-V2 which makes a balance amongspeed, accuracy and stability. Firstly, on the basis of a lightweight backbone,we propose a meta-joint optimization strategy to dynamically regress a smallset of 3DMM parameters, which greatly enhances speed and accuracysimultaneously. To further improve the stability on videos, we present avirtual synthesis method to transform one still image to a short-video whichincorporates in-plane and out-of-plane face moving. On the premise of highaccuracy and stability, 3DDFA-V2 runs at over 50fps on a single CPU core andoutperforms other state-of-the-art heavy models simultaneously. Experiments onseveral challenging datasets validate the efficiency of our method. Pre-trainedmodels and code are available at https://github.com/cleardusk/3DDFA_V2.
Code Repositories
Benchmarks
| Benchmark | Methodology | Metrics |
|---|---|---|
| 3d-face-reconstruction-on-aflw2000-3d | 3DDFA-V2 | Mean NME : 3.56% |
| 3d-face-reconstruction-on-florence | 3DDFA_V2 | Mean NME: 3.56 |
| 3d-face-reconstruction-on-now-benchmark-1 | 3DDFA_V2 | Mean Reconstruction Error (mm): 1.57 Median Reconstruction Error: 1.23 Stdev Reconstruction Error (mm): 1.39 |
| 3d-face-reconstruction-on-realy | 3DDFA-v2 | @cheek: 1.757 (±0.642) @forehead: 2.447 (±0.647) @mouth: 1.597 (±0.478) @nose: 1.903 (±0.517) all: 1.926 |
| 3d-face-reconstruction-on-realy-side-view | 3DDFA-v2 | @cheek: 1.781 (±0.636) @forehead: 2.465 (±0.622) @mouth: 1.642 (±0.501) @nose: 1.883 (±0.499) all: 1.943 |
| 3d-face-reconstruction-on-stirling-hq-fg2018 | 3DDFA_V2 | Mean Reconstruction Error (mm): 1.91 |
| 3d-face-reconstruction-on-stirling-lq-fg2018 | 3DDFA_V2 | Mean Reconstruction Error (mm): 2.10 |
| face-alignment-on-aflw | 3DDFA_V2 | Mean NME: 4.43 |
| face-alignment-on-aflw2000-3d | 3DDFA_V2 | Balanced NME (2D Sparse Alignment): 3.51% Mean NME(3D Dense Alignment): 4.18% |
Build AI with AI
From idea to launch — accelerate your AI development with free AI co-coding, out-of-the-box environment and best price of GPUs.