Read the FAQs and search the forum before posting a new topic.
This forum is for discussing tips and understanding the process involved with Training a Faceswap model.
If you have found a bug are having issues with the Training process not working, then you should post in the Training Support forum.
Please mark any answers that fixed your problems so others can find the solutions.
So basically, I have A(video source) and B (swap source) and I'm on the 12k iteration and realised my training was wrong as it should be the other way around. So I basically stopped the training and saved and redid the command but realised my model training started all over again however it resumed from the 12k iteration but the pictures were blurry. Anyway I can resume training? or do I have to start all over again?
I'm using CLI not GUI btw.
If you accidentally swapped it, I'd suggest restoring from an earlier snapshot and continuing from that instead of continuing with it swapped. You can then tell convert at the end to swap the model and still get your results in the right direction. If you're on Dlight or Realface then you'll actually need to restart since these models work asymmetrically.
If you accidentally swapped it, I'd suggest restoring from an earlier snapshot and continuing from that instead of continuing with it swapped. You can then tell convert at the end to swap the model and still get your results in the right direction. If you're on Dlight or Realface then you'll actually need to restart since these models work asymmetrically.
bryanlyon wrote: ↑Thu Feb 13, 2020 7:50 pmIf you accidentally swapped it, I'd suggest restoring from an earlier snapshot and continuing from that instead of continuing with it swapped. You can then tell convert at the end to swap the model and still get your results in the right direction. If you're on Dlight or Realface then you'll actually need to restart since these models work asymmetrically.
thanks. may I know how do I go about restoring from an earlier snapshot?
never mind. I think I've got it. I ran the GUI and used the tool and pointed to my model dir and let it restore my backup and ran back the re-swapped back the A->B . preview is showing and it's back to normal.
If this is what you meant then thanks! or if not, please do guide me. thanks!
edit: could you explain how it knows which one to restore? and if so, when did it backup?
The training process saves a backup copy of the model each time the loss goes down. The loss wouldn't have caught up from a short train while reversed so wouldn't overwrite the previous best score. There are also snapshots saved separately every (by default) 25000 iterations.