Faceswap Forum

I am trying to do a faceswap of Conor McGregor (faceA)

And another person (faceB)

I am trying to swap into this video: https://www.youtube.com/watch?app=desktop&v=FHwTKXrbEWU

A sample video of faceB is here:
https://www.dropbox.com/s/szca8xr0if7zq ... B.mp4?dl=0

For faceA I have extracted 1111 faces
For faceB I have extracted 8147 faces

I have a machine with 64GB Memory Intel i7-6850k
Four NVIDIA 1080Ti (11GB each) GPU's

Here is some of my workflow so far:

Extract:
Detector: S3Fd
Aligner: Fan
Masker: Bisenet-Fp, Unet-Dfl, Vgg-Clear
Normalization: Hist
Size: 512
Extract Every N: 1

I used the Sort tool, sorted faces, removed the ones that were not good and then updated the alignments files.

Train:
Trainer: Original
Batch Size: 192
Distributed checked

So far I just have the one video for Conor McGregor (faceA), but obvioulsy I can get many of these via YouTube and other sites. I realize I need more videos of him.

Here is a link to the videos I have for faceB. I am going to make more. Any advice on anything I should do in making my new videos for faceB?

https://www.dropbox.com/sh/897pcfk3zfqw ... svLCa?dl=0

Thanks for any tips! So far, after training for about 16 hours (30,000 iterations of batch size 192), I am not liking the results, it really doesn't look much different from the original video. I realize I need more content, but aside from that I am trying to validate other parts of my work flow.

Thank you for any help!

My first suggestion is to make sure you've got a lot of variety. I can't stress the importance of variety enough. Try to get a wide variety of lighting, expression, and pose. It's so incredibly important to get a wide variety as without that you will never get a good result.

Especially with Face A only having 1100 images, I think you need to search out some variety there. Look for more content with your Face A to make sure you have a huge variety of various faces.

If you want to post a screenshot of your training previews we can usually see pretty easily if you're missing a lot of necessary data.

I would add, in terms of workflow:

Don't bother adding any maskers during extract time. It just slows down extraction. Do a "first pass" extraction, which is used purely to identify and extract faces from a video. Sort the faces (one source at a time) and clean up the face set. Clean the alignments file with the remove-faces job. Then use the manual tool to fix up any badly aligned faces. Fixing up the alignments in this way deletes any NN based masks (as the face image will now have changed, so the mask is obsoleted).

Once this is done, you can use the mask tool to add any masks you require to the alignments file and re-extract the faces for your training set.

Original Model, with a batch size of 192??

Not sure that's gonna get you very good results. I suggest Villain or DFL-SAE. I get the best results from them.