Questions on Multiple Sources

Want to know about the Faceswap's Face Extraction process? Got tips, ideas or just want to learn about how it all works? Then this is the place for you


Forum rules

Read the FAQs and search the forum before posting a new topic.

This forum is for discussing tips and understanding the process involved for Extracting and preparing face sets for training a model in Faceswap.

If you have found a bug are having issues with the Extraction process not working, then you should post in the Extract Support forum.

Please mark any answers that fixed your problems so others can find the solutions.

Locked
User avatar
Signal
Posts: 5
Joined: Wed Jul 07, 2021 1:34 pm
Been thanked: 2 times

Questions on Multiple Sources

Post by Signal »

I have been searching and reading all I can find on this, but I am still a bit confused and hoping someone can help.

Lets say I have 4 videos to extract FaceB from. So I extract each one, and end up with all of them in a single output directory, and I have 4 separate alignment files.

So then I can run a Tools->Sort, and sort by faces, delete the faces I don't want, and then I can Remove Faces from each of the 4 alignment files 1 by 1.

Then my understanding is I can merge the alignment files.

But if these are Face B faces, then what is the purpose of the alignment files, my understanding is that for the purpose of Training, the alignments are stored in the PNG's of the faces files themselves, so what is the alignment file being used for?

Is the workflow I mention the right way? Or should I merge each of the alignment files first, and then remove missing?

Is it correct that for training, in order to align the faces, it uses the landmarks which are stored in the PNG files meta data? If so, what is the alignment file doing for training?

And for conversion, the only alignment file that matters is the alignment file of the video you are swapping into correct?

Thank you

Brian

User avatar
torzdf
Posts: 2651
Joined: Fri Jul 12, 2019 12:53 am
Answers: 159
Has thanked: 129 times
Been thanked: 622 times

Re: Questions on Multiple Sources

Post by torzdf »

Signal wrote: Thu Aug 26, 2021 6:57 pm

Lets say I have 4 videos to extract FaceB from. So I extract each one, and end up with all of them in a single output directory, and I have 4 separate alignment files.

So then I can run a Tools->Sort, and sort by faces, delete the faces I don't want, and then I can Remove Faces from each of the 4 alignment files 1 by 1.

You should extract to separate folders, then move the final faces that you want for training to one folder at train time. It may work the way you are suggesting, and if it does, great, but it was never designed this way.

Signal wrote: Thu Aug 26, 2021 6:57 pm

Then my understanding is I can merge the alignment files.

But if these are Face B faces, then what is the purpose of the alignment files, my understanding is that for the purpose of Training, the alignments are stored in the PNG's of the faces files themselves, so what is the alignment file being used for?

Merging alignments files was removed a while ago. In the past alignments files were also required for training, hence merging was necessary to be able to compile training images from multiple sources. Now the alignment data is stored in the PNG header, so merging of files is no longer required. If you have seen any reference to merging alignments files in the official guides (i.e. those I wrote) please let me know so that I can amend.

The alignment file remains the master document though. At some point I will make it easier to work with alignments stored in PNG headers, but it was not originally designed that way, so it will take some time (which I do not have a lot of at the moment). Ultimately the alignment file is your master document. It is linked to a video source. If you want to be able to edit the landmarks/add new masks/re-extract faces, then you need to keep this alignments file. If you are "1 and done" (i.e. you are training the model with your faceset but then never intend to use that faceset again), then sure, delete the file. Personally, I prefer to keep the video + alignments file around and re-generate facesets when I need them, rather than keep folders full of faces around.

Is it correct that for training, in order to align the faces, it uses the landmarks which are stored in the PNG files meta data? If so, what is the alignment file doing for training?

As mentioned above, alignment file is not used for training. Indeed, there is no way to pass an alignment file into Faceswap for training any more.

And for conversion, the only alignment file that matters is the alignment file of the video you are swapping into correct?

Correct

My word is final

User avatar
unsoundmethods
Posts: 11
Joined: Sat Nov 27, 2021 5:16 pm
Has thanked: 3 times
Been thanked: 2 times

Re: Questions on Multiple Sources

Post by unsoundmethods »

And for conversion, the only alignment file that matters is the alignment file of the video you are swapping into correct?

Correct

Oh, do I need to run a face extraction on the video I want to swap the face on?
I did originally, but the video in question hasn't got enough useful images for the training so I added extra from interviews, other video's etc. That greatly helped the training, but as I kept adding, the alignment files are all over the place and there is no alignment file for the complete video I want to swap into.

So the two questions;

  • Did I ruin 3 nights of training by adding from multiple unrelated sources ? (Same person though)
    Do I need to run a face extraction on full length the video I want to swap the face on?

Thanks, brand new into the whole face swapping thing here. Good fun though a bit slow, kind of reminds me of calculating fractals in the late 80's

User avatar
torzdf
Posts: 2651
Joined: Fri Jul 12, 2019 12:53 am
Answers: 159
Has thanked: 129 times
Been thanked: 622 times

Re: Questions on Multiple Sources

Post by torzdf »

unsoundmethods wrote: Sat Nov 27, 2021 5:34 pm

Oh, do I need to run a face extraction on the video I want to swap the face on?

Yes. The convert guide (and by extension the extract guide) covers this.

unsoundmethods wrote: Sat Nov 27, 2021 5:34 pm

Did I ruin 3 nights of training by adding from multiple unrelated sources ? (Same person though)

No this is 100% the right thing to do. More sources = more better.

unsoundmethods wrote: Sat Nov 27, 2021 5:34 pm

Do I need to run a face extraction on full length the video I want to swap the face on?[/list]

yes, see above.

My word is final

User avatar
unsoundmethods
Posts: 11
Joined: Sat Nov 27, 2021 5:16 pm
Has thanked: 3 times
Been thanked: 2 times

Re: Questions on Multiple Sources

Post by unsoundmethods »

Excellent, thanks for the reply.

The whole thing seems to work now, adding the extra references works like a charm.
Slow progress though on my trusty old GTX 970 on a laptop, but great fun.

I started in 1982 with my 16kb Atari, 40 years later, with 128GB RAM on a 32core machine I still have to wait.

User avatar
Morkerflat
Posts: 5
Joined: Mon Jan 17, 2022 2:37 pm

Extracting from multiple videos + some random images

Post by Morkerflat »

I found this question asked here in the forums but I didn't quite understand the answer and also I wanna know about images too.

So how do I make a FaceB folder from multiple videos? Each time I extract I get an allignments file. I believe merging is no longer required so what do I do exactly?

Also in addition to having several videos of the person I also have some random images too, how do I combine them all into one folder?

User avatar
torzdf
Posts: 2651
Joined: Fri Jul 12, 2019 12:53 am
Answers: 159
Has thanked: 129 times
Been thanked: 622 times

Re: Questions on Multiple Sources

Post by torzdf »

Please see above, if you're still now sure, feel free to ask for clarification

My word is final

Locked