The place to discuss Faceswap and Deepfakes
https://forum.faceswap.dev/
Am I correct to assume that for faces with obstructions, the landmark points “behind” the obstruction are completely irrelevant? This is all assuming that your mask is setup to ignore that area. So I don’t need to bother about setting up the points to represent what the structure of the face looks like behind obstructions?
No they are not. The extracted face is "aligned" based on the position of the face landmarks. Ideally the landmarks will appear in the correct position as if the face were to be fully visible.
How then can we handle landmarks that are thrown off by an arm moving in front of the face?
Generally when you move the landmark box around, tighten the points to the temples and chin, the FAN aligner does a really good job of finding a face. But when an obstruction moves in, the alignment points go all crazy - and you end up with the mouth alignments randomly shooting all over the place.
This is also the case when a face is in profile: sometimes it will find an eye brow, maybe an eye, but generally the whole things just goes nuts.
It would be create if there was a way to delete certain landmarks - thus telling the system to not worry about that part of the face.
Anyways, any suggestions would be appreciated!
Yeah, it doesn't deal with obstructions well. Normally copying alignments from previous or next face and then moving the extract box is enough to remedy this.