Skip to content

Commit c382ca6

Browse files
Update README.md
1 parent aff2936 commit c382ca6

File tree

1 file changed

+16
-3
lines changed

1 file changed

+16
-3
lines changed

README.md

+16-3
Original file line numberDiff line numberDiff line change
@@ -13,13 +13,18 @@ We test our method on two commonly used facial expression datasets, [**CoMA**](h
1313

1414
### 3.1 Label control
1515

16+
We perform a conditional generation according to the expression label y.
17+
1618
Examples
1719

1820

1921
<img src="results/angry.gif" width="30%" height="30%" /> <img src="results/eyebrow.gif" width="30%" height="30%" /> <img src="results/mouth_extreme.gif" width="30%" height="30%" /> <img src="results/disgust.gif" width="30%" height="30%" /> <img src="results/mouth_open.gif" width="30%" height="30%" /> <img src="results/lips_up_2.gif" width="30%" height="30%" />
2022

2123

2224
### 3.2 Text control
25+
We perform a conditional generation according to a text. Note that the input texts “disgust high smile” and “angry mouth down” are the combinations of two terms used for training. For instance, “disgust high smile” is a new description that hasn’t been seen before, which combines “disgust” and “high smile”.
26+
27+
2328

2429
Text to expression examples:
2530

@@ -32,6 +37,8 @@ Text to expression examples:
3237

3338
### 3.3 Sequence filling
3439

40+
Similarly to inpainting whose purpose is to predict missing pixels of an image using a mask region as a condition, this task aims to predict missing frames of a temporal sequence by leveraging known frames as a condition.
41+
3542
#### Filling from the beginning.
3643
<img src="results/ffb_1.gif" width="30%" height="30%" /> <img src="results/ffb_2.gif" width="30%" height="30%" />
3744

@@ -44,8 +51,13 @@ Text to expression examples:
4451

4552
### 3.4 Diversity
4653

54+
The specific aim of the 3D facial animation generation is to learn a model that can generate facial expressions that are realistic, appearance-
55+
preserving, rich in diversity, with various ways to condition it.
56+
4757
#### Diversity of label control
4858

59+
The diversity of the generated sequences in terms of expression is shown hereafter. The meshes are obtained by retargeting the expression of the generated 𝑥0 on the same neutral faces.
60+
4961
mouth side
5062

5163
<img src="results/mouth_side_d.gif" width="50%" height="50%" />
@@ -55,6 +67,8 @@ mouth up
5567
<img src="results/mouth_up_d.gif" width="50%" height="50%" />
5668

5769
#### Diversity of Geometry-adaptive generation
70+
In the Geometry-adaptive generation task, we generate a facial expression from a given facial anatomy. This task can also be guided by a classifier. In order to benefit from the consistent and quality expressions adapted to the facial morphology by the DDPM, one can extract a landmark set 𝐿 from a mesh 𝑀, perform the geometry-adaptive task on it to generate a sequence involving 𝐿, and retarget it to 𝑀 by the landmark-guided mesh deformation. We show hereafter the diversity of the generated sequences.
71+
5872

5973
eyebrow
6074

@@ -68,9 +82,6 @@ lips up
6882
### 3.5 Comparison
6983
#### Label control
7084

71-
We perform a conditional generation according to the expression label y.
72-
Conditioning the reverse process of an unconditional DDPM is achieved here by using classifier-guidance.
73-
7485
"high smile"
7586

7687
<img src="results/comp_high_smile.gif" width="50%" height="50%" />
@@ -91,6 +102,8 @@ Conditioning the reverse process of an unconditional DDPM is achieved here by us
91102

92103
### 3.6 Expression retargeting
93104

105+
The landmark sequence taken from a full sequence of the CoMA dataset is retargeted onto several facial meshes.
106+
94107
<img src="results/exp_retarget.gif" width="50%" height="50%" />
95108

96109
## 4. Code

0 commit comments

Comments
 (0)