Two consistent problems - Blur and distortion when head is leaning back

Getting errors or found a bug when converting faces from a trained model? Post about them here


Forum rules

Read the FAQs and search the forum before posting a new topic.

This forum is for reporting errors with the Convert process. If you want to get tips, or better understand the Convert process, then you should look in the Convert Discussion forum.

Please mark any answers that fixed your problems so others can find the solutions.

Locked
User avatar
unkempt
Posts: 81
Joined: Wed Dec 28, 2022 2:09 pm
Has thanked: 1 time
Been thanked: 9 times

Two consistent problems - Blur and distortion when head is leaning back

Post by unkempt »

I have a good training set as far as I know, but I've noticed on several faceswaps now that if they model leans their head back more than a tiny bit, the face becomes wildly distorted. In my replacement face folder, I DO Have faces where they're leaned back and the alignments seem ok (the nose is just a few lines so I have no way to tell for sure). Am I not doing enough iterations? I'm pretty sure I've gone up to 400k or more in some cases.

Similar for blur. I've noticed that some faces are really clear, but others are blurry. Mostly the full screen face where the camera got ... well... in their face. Is that just a resolution problem?

User avatar
torzdf
Posts: 2665
Joined: Fri Jul 12, 2019 12:53 am
Answers: 159
Has thanked: 131 times
Been thanked: 625 times

Re: Two consistent problems - Blur and distortion when head is leaning back

Post by torzdf »

Ultimately this kind of issue will come down to one of 2 reasons:

  1. Badly aligned data. As you say yourself, it can be hard to evaluate whether extreme angles are well aligned
  2. Lack of data. Ultimately the data which is best represented will develop faster than badly represented data. Extreme angles tend to appear less in the dataset, so take longer to learn.

My word is final

User avatar
unkempt
Posts: 81
Joined: Wed Dec 28, 2022 2:09 pm
Has thanked: 1 time
Been thanked: 9 times

Re: Two consistent problems - Blur and distortion when head is leaning back

Post by unkempt »

What do I do about the blur though? I'm pretty sure I'm good with 512x512 source faces that are clear for the most part. Is there a better model for sharpness? Or do I need to push upwards of 400k on training for that?

User avatar
unkempt
Posts: 81
Joined: Wed Dec 28, 2022 2:09 pm
Has thanked: 1 time
Been thanked: 9 times

Re: Two consistent problems - Blur and distortion when head is leaning back

Post by unkempt »

Would it help if I used more higher resolution photos than medium res frames of video?

User avatar
torzdf
Posts: 2665
Joined: Fri Jul 12, 2019 12:53 am
Answers: 159
Has thanked: 131 times
Been thanked: 625 times

Re: Two consistent problems - Blur and distortion when head is leaning back

Post by torzdf »

I'm not really sure what I can add about blur beyond what has been already said. 99% of the time this issue is with data. Some steps to improve things are detailed here: app.php/faqpage#f4r1

My word is final

Locked