How to correct the dreaded micro-jitter?

Want to understand the training process better? Got tips for which model to use and when? This is the place for you


Forum rules

Read the FAQs and search the forum before posting a new topic.

This forum is for discussing tips and understanding the process involved with Training a Faceswap model.

If you have found a bug are having issues with the Training process not working, then you should post in the Training Support forum.

Please mark any answers that fixed your problems so others can find the solutions.

Post Reply
User avatar
JDBAU
Posts: 15
Joined: Wed Apr 24, 2024 6:00 pm
Has thanked: 4 times
Been thanked: 2 times

How to correct the dreaded micro-jitter?

Post by JDBAU »

I always tick re-feed (10x) and realign when extracting but sometimes I still get bad micro-jitter.

In my experience manually tying to fix alignments often has little to no affect, especially if I'm already deep into training when I noticed a face is wobbling slightly in a particular scene.

Does anyone have suggestions on how to best avoid micro jitter or fix it?

User avatar
torzdf
Posts: 2752
Joined: Fri Jul 12, 2019 12:53 am
Answers: 159
Has thanked: 141 times
Been thanked: 643 times

Re: How to correct the dreaded micro-jitter?

Post by torzdf »

I have not had jitter in convert for a long time... It may be down to several things:

  • The mask you are using
  • Overtraining
  • Lack of data

Which mask are you using?

My word is final

User avatar
JDBAU
Posts: 15
Joined: Wed Apr 24, 2024 6:00 pm
Has thanked: 4 times
Been thanked: 2 times

Re: How to correct the dreaded micro-jitter?

Post by JDBAU »

Using Bisenet-Fp.

When extracting I might have had normalization set to none. Maybe changing that would help. Everything else is default.

I'm only at 400k iterations. Target folder has 1.9k images.

User avatar
torzdf
Posts: 2752
Joined: Fri Jul 12, 2019 12:53 am
Answers: 159
Has thanked: 141 times
Been thanked: 643 times

Re: How to correct the dreaded micro-jitter?

Post by torzdf »

Ok, that mask is fine (landmark based masks can be more susceptible to it).

It is extremely unlikely to be extraction related (the model is built to handle a degree of jitter), and is far more likely model/data related

My word is final

User avatar
JDBAU
Posts: 15
Joined: Wed Apr 24, 2024 6:00 pm
Has thanked: 4 times
Been thanked: 2 times

Re: How to correct the dreaded micro-jitter?

Post by JDBAU »

Ok. I'm currently using Dlf-Sae 256.

Most of the scenes are fine, it's just a couple where the swap looks great but there is micro-jitter.

Do you have any tips for avoiding jitter in difficult situations eg, lighting is bad or the target face is far away and very small?

I haven't tried messing with settings yet.

User avatar
torzdf
Posts: 2752
Joined: Fri Jul 12, 2019 12:53 am
Answers: 159
Has thanked: 141 times
Been thanked: 643 times

Re: How to correct the dreaded micro-jitter?

Post by torzdf »

I'm afraid not. It is not an issue that has impacted me for a long time. As I said, it is likely to be data related, so make sure you are well covered off on both sides for the scenes you are struggling with.

My word is final

User avatar
JDBAU
Posts: 15
Joined: Wed Apr 24, 2024 6:00 pm
Has thanked: 4 times
Been thanked: 2 times

Re: How to correct the dreaded micro-jitter?

Post by JDBAU »

Selecting 'no flip' and 'no warp' has all but fixed this issue for me. :D

I trained up to a 1M iterations and still had some faces that were very blurry or had mad jittering. I've now trained another 60k iterations with 'no flip' and 'no warp' and it's made a MASSIVE improvement. Weird. I'm thinking I'll train this model for another 140k iteration to finish it off.

Another thing I discovered is that selecting Re Align during extraction can actually hurt the extraction in that some frames might be skipped altogether (especially profile faces) and not have an alignment written to them. I can upload the video I noticed this happening on for testing purposes if needed.

User avatar
JimCrews
Posts: 1
Joined: Fri Aug 30, 2024 11:35 pm

Re: How to correct the dreaded micro-jitter?

Post by JimCrews »

I'm having that micro jitter and the bad blurry face. I had 250,000 iterations. If I select "no flip' and 'no warp' and wait until I have over one million iterations will the blurry face really go away?

User avatar
JDBAU
Posts: 15
Joined: Wed Apr 24, 2024 6:00 pm
Has thanked: 4 times
Been thanked: 2 times

Re: How to correct the dreaded micro-jitter?

Post by JDBAU »

JimCrews wrote: Tue Sep 03, 2024 3:21 am

I'm having that micro jitter and the bad blurry face. I had 250,000 iterations. If I select "no flip' and 'no warp' and wait until I have over one million iterations will the blurry face really go away?

No, it should only take a few thousand iterations before you see a big improvement.

If the face is a total mess due to a scrambled face alignment you will need to fix the alignment manually via tools > manual.

If the face is recognizable but its blurry / low resolution or shaky, try clicking "no flip' and 'no warp.' This should drastically improve how it looks after 5 to 10k iterations.

Post Reply