Is it possible to train on two faces in a video? Can your replace them both?

If training is failing to start, and you are not receiving an error message telling you what to do, tell us about it here


Forum rules

Read the FAQs and search the forum before posting a new topic.

This forum is for reporting errors with the Training process. If you want to get tips, or better understand the Training process, then you should look in the Training Discussion forum.

Please mark any answers that fixed your problems so others can find the solutions.

Locked
User avatar
unkempt
Posts: 81
Joined: Wed Dec 28, 2022 2:09 pm
Has thanked: 1 time
Been thanked: 9 times

Is it possible to train on two faces in a video? Can your replace them both?

Post by unkempt »

If they were the same face (twins), would that make a difference? Bottom line, is it possible to train on two faces either separately or at once to convert multiple faces in one go?

User avatar
torzdf
Posts: 2636
Joined: Fri Jul 12, 2019 12:53 am
Answers: 155
Has thanked: 128 times
Been thanked: 614 times

Re: Is it possible to train on two faces in a video? Can your replace them both?

Post by torzdf »

In this hypothetical, if they were identical twins, with no obvious differentiating features, then they would be to all intents and purposes the same face.

The process for swapping multiple faces in the frame is, at a high level, as follows (we'll call our identities 'Bob' and 'Sue')

Extract

  • Extract from the source video
  • Clean the alignments files for all misaligns and identities that are not 'Bob' or 'Sue'
  • Copy the alignments file
  • On version 1 of the alignments file delete all the 'Sue' faces and clean the file. This is now your 'Bob' alignments
  • On version 2 of the alignments file delete all the 'Bob' faces and clean the file. This is now your 'Sue' alignments

Train

  • Train 2 models, one for Bob, one for Sue

Convert

  • Run convert against the 'Bob' alignments with the 'Bob' model.
  • Use the converted video to run convert again against the 'Sue' alignments with the 'Sue' model.
Last edited by torzdf on Sun Feb 05, 2023 3:02 pm, edited 1 time in total.

My word is final

Locked