Page 1 of 1

Swapping long videos

Posted: Thu Mar 16, 2023 8:54 am
by jcarl

If swapping the same A and B a 30 minute show... or an entire season of 30 minute shows, should I reuse a single model, or create multiple models?

Would each episode have to be split into smaller scenes (assuming 50k+ A-Faces per episode), or just trained all at the same time?

If using the same model, would I just replace the entire A-side of faces when training a new scene, or episode?

If using separate models, would load/freeze weights be used?


Re: Swapping long videos

Posted: Thu Mar 16, 2023 1:46 pm
by torzdf

I would use a single model for this. You train to swap the faces of 2 identities. You don't train to swap for specific scenes.

You would train your A side on faces taken from all scenes (and any other sources you can obtain).

You would create and clean an alignments file for each episode you wish to convert.

Then you would just run convert on each of these episodes with your 1 trained model


Re: Swapping long videos

Posted: Thu Mar 16, 2023 4:38 pm
by jcarl

Thank you for the reply.

Are there any negative repercussions or hard limits of how many images to with at one time (50,000-500,000 faces)?

I remember the FAQ recommending 500-5,000 faces per side. I was concerned about trying to train with 10-100x the number.

When a new season of a show is released after the model is well trained, is it better to add the new season faces to the existing data set, or replace the old with the new?


Re: Swapping long videos

Posted: Thu Mar 16, 2023 7:07 pm
by bryanlyon

I don't recommend training with such large datasets. Not for hard reasons, just it's a huge pain to clean.

I also don't recommend converting whole TV episodes all at once. Much better to do it scene-by-scene. Then you can ensure that the quality level is high for each scene before moving on. There are a lot of tools that can split your video by scene/cut/transition that can assist with that.