Consistently horrifying smear face when head is tilted back

If training is failing to start, and you are not receiving an error message telling you what to do, tell us about it here


Forum rules

Read the FAQs and search the forum before posting a new topic.

This forum is for reporting errors with the Training process. If you want to get tips, or better understand the Training process, then you should look in the Training Discussion forum.

Please mark any answers that fixed your problems so others can find the solutions.

Locked
User avatar
unkempt
Posts: 81
Joined: Wed Dec 28, 2022 2:09 pm
Has thanked: 1 time
Been thanked: 9 times

Consistently horrifying smear face when head is tilted back

Post by unkempt »

Image

This pic is from a transition where the head is tilted pretty far back. On the right, you can see the distended blob that it became which is what always happens when I do faceswaps from any source video. I doublechecked my face b pics and alignments and they seem fine. The alignments on the source seem fine. I don't know what's going on. I've tried Realface (600000), Villain (400000), and Dlight (250000), but no luck.

Last edited by unkempt on Wed Aug 02, 2023 1:06 pm, edited 2 times in total.
User avatar
unkempt
Posts: 81
Joined: Wed Dec 28, 2022 2:09 pm
Has thanked: 1 time
Been thanked: 9 times

Re: Consistently horrifying smear face when head is tilted back

Post by unkempt »

I'm also having a weird problem with one eye being open and the other closed...

Image

At several points during this video (haven't seen this issue before - or wasn't too noticable), the eyes are looking in the wrong direction or one eye is weirdly closed when it shouldn't be.

User avatar
torzdf
Posts: 2752
Joined: Fri Jul 12, 2019 12:53 am
Answers: 159
Has thanked: 141 times
Been thanked: 643 times

Re: Consistently horrifying smear face when head is tilted back

Post by torzdf »

unkempt wrote: Wed Aug 02, 2023 1:05 pm

Image

This occurs because you don't have enough/any data in B representing this angle, so the model cannot swap it effectively. If you look, A can recreate this fine, B cannot. That is due to insufficient data covering that pose.

2 options:

  • Get data for B that covers these more extreme poses
  • Avoid using such extreme poses in your final video.

re: eyes. These can be tricky, Some models handle it better than others. I know that dlight + realface can have issues with eyes. It may be down to the unbalanced nature of the models, it may not be.

I don't have such a direct solve for this one. Generally ensuring data covering eye directions wouldn't hurt. However why a model may learn that eye direction is one way for the A side and different for the B side I couldn't say due to the 'black box' nature of Machine Learning

Last edited by torzdf on Thu Aug 03, 2023 10:27 am, edited 1 time in total.

My word is final

User avatar
unkempt
Posts: 81
Joined: Wed Dec 28, 2022 2:09 pm
Has thanked: 1 time
Been thanked: 9 times

Re: Consistently horrifying smear face when head is tilted back

Post by unkempt »

Fair enough. So I need more source material with that angle. You're right that it's model-dependant on the eye thing though. I've only ever seen that dlite.

Locked