Where does the eye direction get stored for the training/conversion?
Posted: Sun Oct 02, 2022 8:10 pm
There is no landmark for the position of the pupils... so how is this parsed in the conversion?
The place to discuss Faceswap and Deepfakes
https://forum.faceswap.dev/
There is no landmark for the position of the pupils... so how is this parsed in the conversion?
It's not provided explicitly to the model. It learns to find it by itself from the data.
To expand on this, whilst alignments are extremely important, they are actually only used for 3 reasons.
1) To be able to generate masks landmark based masks (uses all 68 points)
2) To align faces for training (uses the core central face landmarks. i.e. all landmarks except for the jawline)
3) To tell convert where a face is in the frame (this is actually the same as 2, but from a workflow perspective, it makes sense to separate it to its own item).