Questions on Multiple Sources

Want to know about the Faceswap's Face Extraction process? Got tips, ideas or just want to learn about how it all works? Then this is the place for you


Forum rules

Read the FAQs and search the forum before posting a new topic.

This forum is for discussing tips and understanding the process involved for Extracting and preparing face sets for training a model in Faceswap.

If you have found a bug are having issues with the Extraction process not working, then you should post in the Extract Support forum.

Please mark any answers that fixed your problems so others can find the solutions.

Post Reply
User avatar
Signal
Posts: 5
Joined: Wed Jul 07, 2021 1:34 pm
Been thanked: 1 time

Questions on Multiple Sources

Post by Signal »

I have been searching and reading all I can find on this, but I am still a bit confused and hoping someone can help.

Lets say I have 4 videos to extract FaceB from. So I extract each one, and end up with all of them in a single output directory, and I have 4 separate alignment files.

So then I can run a Tools->Sort, and sort by faces, delete the faces I don't want, and then I can Remove Faces from each of the 4 alignment files 1 by 1.

Then my understanding is I can merge the alignment files.

But if these are Face B faces, then what is the purpose of the alignment files, my understanding is that for the purpose of Training, the alignments are stored in the PNG's of the faces files themselves, so what is the alignment file being used for?

Is the workflow I mention the right way? Or should I merge each of the alignment files first, and then remove missing?

Is it correct that for training, in order to align the faces, it uses the landmarks which are stored in the PNG files meta data? If so, what is the alignment file doing for training?

And for conversion, the only alignment file that matters is the alignment file of the video you are swapping into correct?

Thank you

Brian


User avatar
torzdf
Posts: 1495
Joined: Fri Jul 12, 2019 12:53 am
Answers: 127
Has thanked: 51 times
Been thanked: 287 times

Re: Questions on Multiple Sources

Post by torzdf »

Signal wrote: Thu Aug 26, 2021 6:57 pm

Lets say I have 4 videos to extract FaceB from. So I extract each one, and end up with all of them in a single output directory, and I have 4 separate alignment files.

So then I can run a Tools->Sort, and sort by faces, delete the faces I don't want, and then I can Remove Faces from each of the 4 alignment files 1 by 1.

You should extract to separate folders, then move the final faces that you want for training to one folder at train time. It may work the way you are suggesting, and if it does, great, but it was never designed this way.

Signal wrote: Thu Aug 26, 2021 6:57 pm

Then my understanding is I can merge the alignment files.

But if these are Face B faces, then what is the purpose of the alignment files, my understanding is that for the purpose of Training, the alignments are stored in the PNG's of the faces files themselves, so what is the alignment file being used for?

Merging alignments files was removed a while ago. In the past alignments files were also required for training, hence merging was necessary to be able to compile training images from multiple sources. Now the alignment data is stored in the PNG header, so merging of files is no longer required. If you have seen any reference to merging alignments files in the official guides (i.e. those I wrote) please let me know so that I can amend.

The alignment file remains the master document though. At some point I will make it easier to work with alignments stored in PNG headers, but it was not originally designed that way, so it will take some time (which I do not have a lot of at the moment). Ultimately the alignment file is your master document. It is linked to a video source. If you want to be able to edit the landmarks/add new masks/re-extract faces, then you need to keep this alignments file. If you are "1 and done" (i.e. you are training the model with your faceset but then never intend to use that faceset again), then sure, delete the file. Personally, I prefer to keep the video + alignments file around and re-generate facesets when I need them, rather than keep folders full of faces around.

Is it correct that for training, in order to align the faces, it uses the landmarks which are stored in the PNG files meta data? If so, what is the alignment file doing for training?

As mentioned above, alignment file is not used for training. Indeed, there is no way to pass an alignment file into Faceswap for training any more.

And for conversion, the only alignment file that matters is the alignment file of the video you are swapping into correct?

Correct

My word is final


Post Reply