Faceswap Forum

The last couple of days I've been playing with Faceswap and created a couple of demo videos in which I swap the face of Jeff Bezos with that of Elon Musk. You can see the result here: https://streamable.com/7w1g6.

The videos I used to train the model are:

• https://www.youtube.com/watch?v=guVxubbQQKE

• https://www.youtube.com/watch?v=wsixsRI-Sz4

I extracted frames in which only the face of the subject occurs and randomly selected about 8000 frames from the video. Then I extracted the faces using:

python faceswap.py extract -i <input> -o <output> -D s3fd -A fan -nm hist

The model used for training is the Villain model:

python faceswap.py train -A <A> -ala <A fsa> -B <B> -alb <B fsa> -m <model> -t villain -bs 32 -nl -g 4

I changed the following parameters in train.ini

coverage = 70.0
loss_function = ssim
reflect_padding = True

Training was done on an AWS p3.8xlarge instance containing 4 Tesla V100 GPU's for 10,175 iterations. The lowest average loss dropped to 0.01620 for A and 0.01586 for B.

To me the result still looks more blurry than I expected. Do you feel the same? Can this be caused by the resolution of the faces that are swapped? In the demo video they are around 275x275 pixels while the output of the network is 128x128. Is it possible to train the Villain network with a larger in- and output size? Or do you have any other suggestions to make the result less blurry?

Much appreciated!

Some suggestions:

1: Use more than one video for each face.

The more the better, having just one video per side is going to lead to poor results

2: Use non-youtube videos.

Youtube videos are highly compressed and lose a lot of detail. It's fine to use them as part of your dataset, but it's best if you can find higher quality sources.

3: Find videos with closer views of the faces

Videos with small faces will lead to poor quality results, try to find larger faces where the face takes up more of the shot.

See viewtopic.php?f=6&t=74 where I go more into depth on these points and others to help ensure you get a good dataset.

Thanks for your reply [mention]bryanlyon[/mention]! Can you explain how "close ups" are improving the quality of the swap? The output of the Villain network is 128x128 pixels, so this will be upscaled to the size of the face in the result, right?

And do you think increasing the in- and output size of the Villain network is feasible?

You want the training faces to be at least that 128x or you're training to lower quality faces and you wont get full quality for the swap.

Villain is already a very heavy model. Without some exotic hardware with a lot of vram, it's pretty much as big as it can get.