Question about the conversion process - alignments versus pixels - need to understand

Got questions or tips about the Conversion process? This is the place to discuss them.


Forum rules

Read the FAQs and search the forum before posting a new topic.

This forum is for discussing tips and understanding the process involved with Converting faces from your trained model.

If you are having issues with the Convert process not working as you would expect, then you should post in the Convert Support forum.

Please mark any answers that fixed your problems so others can find the solutions.

Locked
User avatar
martinf
Posts: 27
Joined: Thu Sep 29, 2022 7:58 pm
Been thanked: 3 times

Question about the conversion process - alignments versus pixels - need to understand

Post by martinf »

Does the alignment file simply tell the model where to find the face, then the model analyzes the pixels for the swap... or are the alignments playing a larger role in the swap process? Ie., Does the model look to the alignments for the face generation or only to the underlying video... after the alignments tell it where to find faces?

I ask because there are clearly nuances in facial expressions that are not part of the alignments, leading me to believe that the alignments only serve as a direction as to where to find a face. Or is it a combination?

User avatar
torzdf
Posts: 2649
Joined: Fri Jul 12, 2019 12:53 am
Answers: 159
Has thanked: 128 times
Been thanked: 622 times

Re: Question about the conversion process - alignments versus pixels - need to understand

Post by torzdf »

It's more complicated than you are saying, but yes the alignments file tells the process where the faces are,

It specifically uses the landmarks to do some complicated mathematics to crop the faces from each image and align them before processing through the model, then reversing the process to put the swapped face back in the frame.

My word is final

Locked