Training from multiple sources - how to combine alignments?

Want to understand the training process better? Got tips for which model to use and when? This is the place for you


Forum rules

Read the FAQs and search the forum before posting a new topic.

This forum is for discussing tips and understanding the process involved with Training a Faceswap model.

If you have found a bug are having issues with the Training process not working, then you should post in the Training Support forum.

Please mark any answers that fixed your problems so others can find the solutions.

Locked
User avatar
Replicon
Posts: 50
Joined: Mon Mar 22, 2021 4:24 pm
Been thanked: 2 times

Training from multiple sources - how to combine alignments?

Post by Replicon »

So far, I've been combining source videos using Kdenlive, and then the "multiple alignment files" problem just goes away.

But now, I'm faced with a situation where I want to incorporate a new source into the pool of sources for an identity.

I'm not about to recombine the videos and redo the alignment work, so I figure there should be a way to just have it read multiple alignment files when training.

In the old guide screenshots, I can see there's a "merge" job on the alignments tool, but that doesn't seem to exist anymore.

I had assumed the right way was to just extract everything into the same directory and pass a list of alignment files to the training, but I was reminded the "alignments" option is deprecated.

So, this leads me to two questions:

1) What's the workflow for training from multiple sources (separate videos, for which I've meticulously adjusted separate alignment files)?

2) In general, how come 'train' doesn't actually need an alignment file input? I mean, it MUST need to know the landmarks/alignments for the frames, which it can't just infer from the extracted photos. Does it just infer the alignment file name from the name of the input directory? If so, how do we do multiple alignment files, if the "merge" tool doesn't exist anymore?

User avatar
sp13
Posts: 15
Joined: Sat Apr 10, 2021 12:20 am
Has thanked: 3 times
Been thanked: 4 times

Re: Training from multiple sources - how to combine alignments?

Post by sp13 »

  1. Put all the extracted face .png files (for one particular person) into one directory and use it as the training source

  2. Alignments are stored in header of the .png files

The alignment files are not used for training at all, so there is no way to read multiple files (or even one) and no need to merge them.

User avatar
Replicon
Posts: 50
Joined: Mon Mar 22, 2021 4:24 pm
Been thanked: 2 times

Re: Training from multiple sources - how to combine alignments?

Post by Replicon »

Ahaaaaaaa, that was the missing piece! I didn't realize the alignments were actually stored in the png headers. Ok, the world makes sense again, thank you!

User avatar
Replicon
Posts: 50
Joined: Mon Mar 22, 2021 4:24 pm
Been thanked: 2 times

Re: Training from multiple sources - how to combine alignments?

Post by Replicon »

Oh BTW, I just updated my script to support extracting from multiple sources to the same output directory, but it fails on the second iteration (second source), with:

ERROR: Output faces folder should be empty

Is this by design? Seems we should be able to use the tool to generate a faces folder for one person from multiple sources/alignments. I'll update my script to output them to a temporary location and move the contents to the real location to work around this.

User avatar
torzdf
Posts: 2651
Joined: Fri Jul 12, 2019 12:53 am
Answers: 159
Has thanked: 129 times
Been thanked: 622 times

Re: Training from multiple sources - how to combine alignments?

Post by torzdf »

Yeah, there are arguments for and against being able to extract to the same folder. The current system is a hangover from how Faceswap was originally built. However, from my point of view at least, the inconvenience of having to manually move faces into the same folder is outweighed by the risk of accidentally overwriting faces in an existing folder (if that makes sense).

My word is final

Locked