Thats a thumbnail from my current preview, for the first like 200k iterations it was fine then it started appearing with this distorting red static, anyone know why/how to fix? Can't work out how to load the project at a prior state which would be useful here
If training is failing to start, and you are not receiving an error message telling you what to do, tell us about it here
This forum is for reporting errors with the Training process. If you want to get tips, or better understand the Training process, then you should look in the Training Discussion forum.
Please mark any answers that fixed your problems so others can find the solutions.
This is model corruption (but not corruption like I have seen before).
For this kind of corruption, I would roll back to an earlier snapshot and possibly lower the learning rate a bit.
To roll back to an ealier snapshot, just delete your model folder, copy one of your snapshot folders, and rename the copy to the original model folder name (i.e. remove the
_snapshot_xxxxxx from the end of the folder name) and then continue training.
My word is final