How important is it for the jawline and eyebrows to be perfect, assuming other landmarks are good?

Want to know about the Faceswap's Face Extraction process? Got tips, ideas or just want to learn about how it all works? Then this is the place for you


Forum rules

Read the FAQs and search the forum before posting a new topic.

This forum is for discussing tips and understanding the process involved for Extracting and preparing face sets for training a model in Faceswap.

If you have found a bug are having issues with the Extraction process not working, then you should post in the Extract Support forum.

Please mark any answers that fixed your problems so others can find the solutions.

Locked
User avatar
Replicon
Posts: 50
Joined: Mon Mar 22, 2021 4:24 pm
Been thanked: 2 times

How important is it for the jawline and eyebrows to be perfect, assuming other landmarks are good?

Post by Replicon »

I know the answer is "as perfect as possible", but I'm hoping for some more qualitative answers, to help decide if something will be good enough, or will be a complete mess if I don't spend the hours adjusting it. :)

To give a specific example that I constructed from stock photos for illustration purposes:

Usually, you get a nice extract that looks something like this:

eg_extract_good.png
eg_extract_good.png (77.73 KiB) Viewed 9478 times

But every now and then, every frame from a scene looks kinda like this:

eg_extract_bad.png
eg_extract_bad.png (77.38 KiB) Viewed 9478 times

Will that still create decent results, or will the trainer think the model has a wide face with a hairy ear growing out of her cheek? :)

I'm thinking it's a gradual shift, so that's why I'm asking... how bad does this kind of thing need to be before it starts to really matter, assuming I'm not trying to convince someone that it's real?

In a similar vein, sometimes everything looks perfect, EXCEPT the eyebrows are like at the top of their forehead... how much of an impact will that have overall?

EDIT TO ADD: Given that the face that gets converted isn't really shaped like the actual mask or landmark lines (looks more like a square), I'm hoping the eyes and mouth are the "high priority" landmarks, and the rest of the image that would land outside the target's face area just gets clipped, so the bad edges don't matter that much.

User avatar
torzdf
Posts: 2651
Joined: Fri Jul 12, 2019 12:53 am
Answers: 159
Has thanked: 129 times
Been thanked: 622 times

Re: How important is it for the jawline and eyebrows to be perfect, assuming other landmarks are good?

Post by torzdf »

If you are using a landmarks based mask (components/extended), then it matters a lot more, as the mask will be wrong if the landmarks are wrong.

If you are not using a landmarks based mask, you can disregard the jaw entirely. Only the core area landmarks are used for alignment (i.e. all landmarks except for the jaw).

"off" landmarks can effect the alignment in unexpected ways. Generally I'll fix a face with "off" landmarks within the manual tool and watch the face preview to see how much it impact the alignment of the extracted face. If it's significant, I will fix all those which are off, if the change is minimal I will not worry about it too much.

Correctly aligned faces = better training and better results.

My word is final

User avatar
Replicon
Posts: 50
Joined: Mon Mar 22, 2021 4:24 pm
Been thanked: 2 times

Re: How important is it for the jawline and eyebrows to be perfect, assuming other landmarks are good?

Post by Replicon »

Aah ok. Then I should probably keep fixing them, since I'm basically defaulting to extended.

It's funny though, even with a well-defined chin, videos converted with extended tend to cut off the bottom just below the mouth, rather than including the full contour of the chin.

User avatar
torzdf
Posts: 2651
Joined: Fri Jul 12, 2019 12:53 am
Answers: 159
Has thanked: 129 times
Been thanked: 622 times

Re: How important is it for the jawline and eyebrows to be perfect, assuming other landmarks are good?

Post by torzdf »

Replicon wrote: Wed Jul 28, 2021 1:21 am

It's funny though, even with a well-defined chin, videos converted with extended tend to cut off the bottom just below the mouth, rather than including the full contour of the chin.

That will depend on the centering used for training, the coverage used, and how much erosion is applied to the mask at convert time.

My word is final

Locked