Page 1 of 1

empty face

Posted: Mon Aug 08, 2022 4:01 pm
by damingxing

I came across an issue: the face of the output video is empty. Does anyone happen to know why and how we can mitigate this from happening? I think this happened in the training process since the images from the extract are fine. Is it possible that there is a vram leakage? (not an expert, just guessing)

I have used the same hyperparameters and settings to successfully create the same videos (source and target) before. The issue did not appear.

I am attaching an example from the output video and the last preview.

Thank you!

example from the output video
example from the output video
image.png (213.77 KiB) Viewed 1746 times
example from the last preview
example from the last preview
empty face1.jpg (39.65 KiB) Viewed 1746 times

Re: empty face

Posted: Mon Aug 08, 2022 5:48 pm
by torzdf

Corrupted model.

Lower learning rate + rollback 50k


Re: empty face

Posted: Fri Aug 12, 2022 6:49 pm
by damingxing

Do you happen to know what may cause this? This recently has been happening quite often on my side. Thanks.


Re: empty face

Posted: Sun Aug 14, 2022 9:28 am
by torzdf

It's model collapse. It happens in AI sadly.


Re: empty face

Posted: Sun Aug 14, 2022 1:14 pm
by damingxing

thanks for the response. I realized there are times the model folder was empty, and there is 'model-nf" folder has .h5 and state status files. Do you happen to know why that is? thanks


Re: empty face

Posted: Sun Aug 14, 2022 1:16 pm
by torzdf

No, I've never heard of that before, I'm afraid. Sounds very strange indeed.

Another thing to check for (re: model corruption) is to make sure there are no overclocks (including factory overclocks) on your GPU, as this can cause corruption.