Please give me advise on my approach

Want to understand the training process better? Got tips for which model to use and when? This is the place for you


Forum rules

Read the FAQs and search the forum before posting a new topic.

This forum is for discussing tips and understanding the process involved with Training a Faceswap model.

If you have found a bug are having issues with the Training process not working, then you should post in the Training Support forum.

Please mark any answers that fixed your problems so others can find the solutions.

Locked
User avatar
Signal
Posts: 5
Joined: Wed Jul 07, 2021 1:34 pm
Been thanked: 2 times

Please give me advise on my approach

Post by Signal »

I am trying to do a faceswap of Conor McGregor (faceA)
Image

And another person (faceB)
Image

I am trying to swap into this video: https://www.youtube.com/watch?app=desktop&v=FHwTKXrbEWU

A sample video of faceB is here:
https://www.dropbox.com/s/szca8xr0if7zq ... B.mp4?dl=0

For faceA I have extracted 1111 faces
For faceB I have extracted 8147 faces

I have a machine with 64GB Memory Intel i7-6850k
Four NVIDIA 1080Ti (11GB each) GPU's

Here is some of my workflow so far:

Extract:
Detector: S3Fd
Aligner: Fan
Masker: Bisenet-Fp, Unet-Dfl, Vgg-Clear
Normalization: Hist
Size: 512
Extract Every N: 1

I used the Sort tool, sorted faces, removed the ones that were not good and then updated the alignments files.

Train:
Trainer: Original
Batch Size: 192
Distributed checked

So far I just have the one video for Conor McGregor (faceA), but obvioulsy I can get many of these via YouTube and other sites. I realize I need more videos of him.

Here is a link to the videos I have for faceB. I am going to make more. Any advice on anything I should do in making my new videos for faceB?

https://www.dropbox.com/sh/897pcfk3zfqw ... svLCa?dl=0

Thanks for any tips! So far, after training for about 16 hours (30,000 iterations of batch size 192), I am not liking the results, it really doesn't look much different from the original video. I realize I need more content, but aside from that I am trying to validate other parts of my work flow.

Thank you for any help!

User avatar
bryanlyon
Site Admin
Posts: 793
Joined: Fri Jul 12, 2019 12:49 am
Answers: 44
Location: San Francisco
Has thanked: 4 times
Been thanked: 218 times
Contact:

Re: Please give me advise on my approach

Post by bryanlyon »

My first suggestion is to make sure you've got a lot of variety. I can't stress the importance of variety enough. Try to get a wide variety of lighting, expression, and pose. It's so incredibly important to get a wide variety as without that you will never get a good result.

Especially with Face A only having 1100 images, I think you need to search out some variety there. Look for more content with your Face A to make sure you have a huge variety of various faces.

If you want to post a screenshot of your training previews we can usually see pretty easily if you're missing a lot of necessary data.

User avatar
torzdf
Posts: 2649
Joined: Fri Jul 12, 2019 12:53 am
Answers: 159
Has thanked: 128 times
Been thanked: 622 times

Re: Please give me advise on my approach

Post by torzdf »

I would add, in terms of workflow:

Don't bother adding any maskers during extract time. It just slows down extraction. Do a "first pass" extraction, which is used purely to identify and extract faces from a video. Sort the faces (one source at a time) and clean up the face set. Clean the alignments file with the remove-faces job. Then use the manual tool to fix up any badly aligned faces. Fixing up the alignments in this way deletes any NN based masks (as the face image will now have changed, so the mask is obsoleted).

Once this is done, you can use the mask tool to add any masks you require to the alignments file and re-extract the faces for your training set.

My word is final

User avatar
dheinz70
Posts: 43
Joined: Sat Aug 15, 2020 2:43 am
Has thanked: 4 times

Re: Please give me advise on my approach

Post by dheinz70 »

Original Model, with a batch size of 192??

Not sure that's gonna get you very good results. I suggest Villain or DFL-SAE. I get the best results from them.

Locked