More A Faces than source video

Want to know about the Faceswap's Face Extraction process? Got tips, ideas or just want to learn about how it all works? Then this is the place for you


Forum rules

Read the FAQs and search the forum before posting a new topic.

This forum is for discussing tips and understanding the process involved for Extracting and preparing face sets for training a model in Faceswap.

If you have found a bug are having issues with the Extraction process not working, then you should post in the Extract Support forum.

Please mark any answers that fixed your problems so others can find the solutions.

Locked
User avatar
DannyA
Posts: 7
Joined: Mon Aug 31, 2020 10:14 am
Has thanked: 3 times

More A Faces than source video

Post by DannyA »

Hello all.

I've had good results so far, the guides are really good. Apologies if this is the wrong forum section, i have tried searching previous posts.

I have a 2min A video which I have extracted about 3k faces. To get better data for FaceA, can I get pictures from other sources, just for training?

The guide seems to suggest I can get another source and merge it in and update the alignments file. This is just for training, not the final convert.

Thanks

User avatar
bryanlyon
Site Admin
Posts: 793
Joined: Fri Jul 12, 2019 12:49 am
Answers: 44
Location: San Francisco
Has thanked: 4 times
Been thanked: 218 times
Contact:

Re: More A Faces than source video

Post by bryanlyon »

Yes, in fact this is HIGHLY recommended. Get a variety of sources for both your A and B faces.

User avatar
DannyA
Posts: 7
Joined: Mon Aug 31, 2020 10:14 am
Has thanked: 3 times

Re: More A Faces than source video

Post by DannyA »

Please excuse my stupidity. I think I am nearly there. I am slightly confused about the alignments file used for Training vs the alignments file for Converting. Does this work flow sound about right?

(1. Extract a Set of faces from FaceA which will be used in final convert. Purpose - to get some images for training, but mainly to set the landmarks/masks etc in the alignments file

(2. Extract a number of faces from different video sources for FaceA, merge them into one alignments file and clean it up. This file is separate to the previous alignments file generated in step 1. This is for training the model.

(3. Extract a set of faces for FaceB, to be used in final convert and also for training. (less sources available for FaceB for me)

Model trains on training images for FaceA and FaceB - when time to convert I use the original alignments file taken in step 1 for FaceA. Even though I have three alignment files in this example, the model is where the magic happens with the alorigithim.

N.B Can use the fit method to add more faces from Source FaceA video if required,

User avatar
abigflea
Posts: 182
Joined: Sat Feb 22, 2020 10:59 pm
Answers: 2
Has thanked: 20 times
Been thanked: 62 times

Re: More A Faces than source video

Post by abigflea »

I think you almost have it.

If I personally was in your scenario as I understand it:

Have good sources for A, mainly from the video you wish to use. I may try squeezing as many out of it as possible. You can also use images of A from other sources but if you don't that available, it may be fine.

The B face is more important for as many sources as possible, whatever good data (images) you can find.

Landmarks are used to create the mask. "Reasonable" landmarks will lead to reasonable mask, so no big worries for training. It's AI , it will learn. I have personally sat down and made the alignments perfect for 4K images on either side and it didn't make all that much difference for training. So I learned that the hard way.
Then let all that train for a while.

Then for the "to-convert-from-from-A-to-B" video I do a separate extraction.
Every single frame. Get the alignments pretty clean. If you have obstructions you may wish to additionally clean up the generated masks. See how it goes.

Then use the model you trained, with the "to-convert-from-from-A-to-B" video and that nice and clean Alignments file.

I usually make a short video(extract, alignments, mask too)...about 20 sec long, and convert that to see how its looking before I do the full one.

:o I dunno what I'm doing :shock:
2X RTX 3090 : RTX 3080 : RTX: 2060 : 2x RTX 2080 Super : Ghetto 1060

User avatar
DannyA
Posts: 7
Joined: Mon Aug 31, 2020 10:14 am
Has thanked: 3 times

Re: More A Faces than source video

Post by DannyA »

Thank you very much

Locked