Where does the eye direction get stored for the training/conversion?

Want to know about the Faceswap's Face Extraction process? Got tips, ideas or just want to learn about how it all works? Then this is the place for you


Forum rules

Read the FAQs and search the forum before posting a new topic.

This forum is for discussing tips and understanding the process involved for Extracting and preparing face sets for training a model in Faceswap.

If you have found a bug are having issues with the Extraction process not working, then you should post in the Extract Support forum.

Please mark any answers that fixed your problems so others can find the solutions.

Locked
User avatar
martinf
Posts: 27
Joined: Thu Sep 29, 2022 7:58 pm
Been thanked: 3 times

Where does the eye direction get stored for the training/conversion?

Post by martinf »

There is no landmark for the position of the pupils... so how is this parsed in the conversion?

User avatar
bryanlyon
Site Admin
Posts: 793
Joined: Fri Jul 12, 2019 12:49 am
Answers: 44
Location: San Francisco
Has thanked: 4 times
Been thanked: 218 times
Contact:

Re: Where does the eye direction get stored for the training/conversion?

Post by bryanlyon »

It's not provided explicitly to the model. It learns to find it by itself from the data.

User avatar
torzdf
Posts: 2651
Joined: Fri Jul 12, 2019 12:53 am
Answers: 159
Has thanked: 129 times
Been thanked: 622 times

Re: Where does the eye direction get stored for the training/conversion?

Post by torzdf »

To expand on this, whilst alignments are extremely important, they are actually only used for 3 reasons.

1) To be able to generate masks landmark based masks (uses all 68 points)
2) To align faces for training (uses the core central face landmarks. i.e. all landmarks except for the jawline)
3) To tell convert where a face is in the frame (this is actually the same as 2, but from a workflow perspective, it makes sense to separate it to its own item).

My word is final

Locked