I've been trying to train a swap using villain but the model after a few thousand iterations just converges to outputting a completely white face.
Any tips to stop this? Thanks!
Read the FAQs and search the forum before posting a new topic.
This forum is for reporting errors with the Training process. If you want to get tips, or better understand the Training process, then you should look in the Training Discussion forum.
Please mark any answers that fixed your problems so others can find the solutions.
I've been trying to train a swap using villain but the model after a few thousand iterations just converges to outputting a completely white face.
Any tips to stop this? Thanks!
This is not an issue I have seen, but it sounds like the model has collapsed.
Roll back 50k iterations and lower the learning rate
My word is final
Thanks for the reply!
I restarted the whole thing and tried rolling back learning rate to 1e-6 and it still converges to white. I reduced my batch size to 4 as well.
Do you have any other tips? Thanks!
I do not.
At this point, it may be a hardware or data related issue.
Make sure that any overclocks (including stock overclocks) are disabled on your GPU.
Beyond that, I could not say, sorry.
My word is final