Hello.
I tried to change the trainer from DFL-H128 to DLIGHT , but I got this instability. It does not seem to be able to recover.
Any advice?
Read the FAQs and search the forum before posting a new topic.
This forum is for discussing tips and understanding the process involved with Training a Faceswap model.
If you have found a bug are having issues with the Training process not working, then you should post in the Training Support forum.
Please mark any answers that fixed your problems so others can find the solutions.
You can't change trainers mid train (No, like it's actually impossible, you're using the same trainer) but what you're really getting here is probably one of the corruptions that sometimes randomly occurs. You can use the restore tool to restore to the last known good model.
Hi...
I explained myself poorly.
I made a video using DFL h128 and then I RESTARTED from scratch using DLIGHT with the same sets/alignment.
DHL 128 was not bad... but I wanted to compare it with something else.
Things went well at the beginning... then everything exploded.
Just wandering if is there any parameter that can prevent this mess....
Yes, that's reasonable. These "blow ups" do just happen randomly sometimes. Thats why we have the restore tool. Just restore to the last good version and continue training.
If it keeps happening, you might want to reduce your Learning Rate in the train settings.
I was going to add that you might want to double check where your A set is pointing. I got a very similar looking problem when I accidentially changed to set things up on a different project, and then didn't change one of my sets back properly. But then I noticed that the error is increasing, and I think mine decreased. In any event, the ultimate solution is the same, which is use the Restore to bring things back, and if that doesn't work you can also just delete everything in your model folder and copy the contents of the nearest major "25K save" into it.