Hello all,
I am still quite a beginner but I wonder the following:
I have started a test project.
During the training I used the original model with a batch size of 16. Snapshot interval is at 250.
Faceswap is training at about 1k iterations per minute. Is this normal? I am using the RTX 4090, the GPU seems to be quite bored at least according to task manager. So it looks like the available resources are not being used optimally.
Task manager is an awful way to verify GPU usage. See app.php/faqpage#f0r3 for more info.
Original on a 4090 is probably a bit of a bottleneck. Honestly I'd recommend trying out DFaker or Phaze A (stojo preset) instead. They both have much higher quality and can give you better results. Original uses just 64x64 images so it's quite possible that loading and preparing the images is becoming the bottleneck in that situation.
Also, setting a snapshot to every 250 iterations means you're going to fill up your system with snapshots. I highly recommend you keep that at default.
thank you for the quick reply. Of course, this explains why I still get blurry results after over 200k iterations, even though the facial structures are clearly visible.
Is there a way to incorporate the already trained model into the new model?
Or would it be better at this point to train from scratch when the trainer has been changed?
During the training I used the original model with a batch size of 16. Snapshot interval is at 250.
I assume you mean save interval is at 250 (and snapshot interval is at 25,000). Snapshot at 250 would slow things down as you would be taking a full backup copy of your model every 250 iterations.
Faceswap is training at about 1k iterations per minute. Is this normal? I am using the RTX 4090, the GPU seems to be quite bored at least according to task manager. So it looks like the available resources are not being used optimally.
Or am I expecting too much?
Thanks for your assessment
Haha, 1k per minute. Rome wasn't built in a day man.
Patience young grasshopper
It ofcourse all depends on the models, amount of paraments etc.
I personally try and keep the models that i can hit above 100 It's per minute on my 4090, but then again i like complex phaze models
Only if power was cheaper here in the old europe