Page 1 of 1
Should I ever expect convergence? Couple of questions.
Posted: Fri Mar 06, 2020 12:41 pm
by shambler74
Good morning! New to DeepFakes and FaceSwap.py in general. I have a couple questions and was hoping to hear from others in the community about them.
1). Will I ever get convergence?
--- Context: Face A has 7000 images, with multiple light sources and 100's of images from every direction including deep side and up/down. Face B has 2400 images with multiple light sources and not as many as face A side to side/up to down but has a decent amount. Knowing the above, should I expect convergence? To clarify, convergence is when I watch the sample images and none of the swapped faces are blurry, right?
2). Iterations for Original/Villain
--- I've read the other threads and know to use my eyes more than iterations or the graph, but I also read I should expect at least 400k iterations for original before convergence becomes a reality. Is this soft limit still in place? If not, are my eyes the true guiding light, when I see no blurry images from A->B in the sample/preview pane?
3). What type of monster machine is needed to run Villain on > 8 batch size?
--- Looking for more of "what you run and how you run it" in a response here. My main gaming rig is a 64 core AMD Threadripper with an nVidia Titan V. I can run 5k games with no problem, cruch ETH mining at 110MH/s with the unclocked Titan V rails and yet Villain will only run at batch size 8. Any larger and it will OOM me. Is this a VRAM size thing? The titan V has 12 GB of vram so I should be able to do batch sizes of 16 right? Why then does it cop out under that?
Thanks for your time reading this thread!
Re: Should I ever expect convergence? Couple of questions.
Posted: Fri Mar 06, 2020 5:23 pm
by bryanlyon
On your questions:
1: Convergence isn't where the face is blurry, but where the model has learned all that it could. Either because you've hit the limit of your data, or the model. The face could still be blurry.
2: Original will probably converge in a couple days, and just not learn more. Villain can take a week or more to reach convergence. Don't look at iteration count as that's effectively a nonsense number.
3: It's a simple matter of vram. Honestly I'm surprised you can get a BS of 8 on Villain. People tend to struggle with it as it's a very heavy model.
Re: Should I ever expect convergence? Couple of questions.
Posted: Fri Mar 06, 2020 7:35 pm
by tiazgriff
Im running Villain on 1080 with batch size 5 no lowmem and playin Wow at the same time. Starting to look good at 300k iters
Re: Should I ever expect convergence? Couple of questions.
Posted: Sun May 24, 2020 6:34 pm
by Metalstylite
tiazgriff wrote: ↑Fri Mar 06, 2020 7:35 pm
Im running Villain on 1080 with batch size 5 no lowmem and playin Wow at the same time. Starting to look good at 300k iters
I used 1080 too.
But with side A 5k imgs, side B 2k2 imgs, it can handle with bs 4 no lowmem setting and 6 with lowmem.
I dont want trade off speed with bs 8 (memory saving). How can you do it when training and play WoW at sametime ? I can only surf web or view movie only while training. 
Re: Should I ever expect convergence? Couple of questions.
Posted: Sun May 24, 2020 7:04 pm
by bryanlyon
1080 can easily handle larger bs than that, make sure to close chrome or any other applications using vram.