Faceswap Forum

ianstephens

Yea I hear what you're saying. We couldn't get another 3090 (founders) in our machine as space is limited and they are behemoths of a card. Good to hear you have feedback from users of the A6000. When you say "double" batch size/speed do you refer to the exact same GPU as the second? We ha...

ianstephens

Just an update. I've managed to compile two decent data sources for both A/B and extracted at 1535px to run with the new DNY 1024 model. I've noticed a few things. We are running Full Precision @ batch size 4 (the highest we could get even after the initial model build). I'm noticing that there is a...

ianstephens

Yes, you're right - what was I thinking. Was already aware of this padding from previous (smaller) models. Having a slow day (or week) :) I'm going to run extraction @ 1536px. I should be able to provide some examples and images from this test run @ 1024px. I'll let you know once I've got somewhere ...

ianstephens

A quick one... and I will reply to this post above very soon... however...

What input image sizes do you use for your DNY 1024px model? We have been running extracting at 1024px - agree?

ianstephens

Are you trying to train models based on 3D VR sources? I have experience with this. We have worked with the side to be swapped (A-Side) in "fisheye" VR format and the side to be replaced (B-Side) with standard flat lens non-fisheye images/video sources. If this is the case, it's not perfec...

ianstephens

So I've been playing with some models, different settings, etc. I've been using your new DNY models as you are familiar with these and provide a better understanding when reporting back. Testing on a single 3090 (24GB VRAM), I am able to load your 1024 px DNY model @ a batch size of 4 max at FP. I h...

ianstephens

I have done a not inconsiderable amount of testing around NaNs recently, really hoping I would find a solve, and I have drawn a blank. Where they get introduced in a model is inconsistent, although they nearly always seem to appear at some point in the forward pass, which clipping would not be able...

ianstephens

Using mixed precision, I am finding that in some tests even with Learning Rate super low (3e-05 or less) on large models, the only way to mitigate NaN is to set the EE to -4 or less. Learning rate, even super low does not seem to fix NaN but setting the EE at a low number seems to help and mitigate ...

ianstephens

Documentation dictates that not much is gained by enabling mask learning. However, I can see some updates recently focusing on this feature. We currently use Bisenet-FP along with FS weights as masking settings. Sometimes, for example, when a subject's nose is larger/longer than the swaps (especiall...

ianstephens

Just playing about with the recently introduced AutoClip feature. How much effect would this have on NaNs? Strong? Or is it still worth tapering down the EE with mixed-precision training? Does anyone have any feedback on this feature so far with very complex models while using mixed precision? I mig...

ianstephens

Haven't tried it yet but... What would happen if we were to merge two datasets of two completely different faces (subject) into Side B while leaving Side A singular (like normal)? Would the AI essentially create a merge of what it thinks they look like combined? A lovechild so to speak? Or would som...

ianstephens

[mention]torzdf[/mention] - I didn't manage to solve it - it persists. We worked around the issue by running the graph full time in the FaceSwap window and enabling the second optional (separate) preview window. That way there is no need for switching in the FS GUI. We simply leave the graph running...

ianstephens

So the white box preview continues when starting brand new models. However, I have found the strangest workaround. Before starting a brand new model, I simply update Faceswap (Help-->Update Faceswap). There doesn't even need to be an update available, I just simply need to perform this action. Once ...

ianstephens

Where was the sarcasm?

The EfficientNetV2 really is something and the results are fantastic. I would agree with the quoted 5x-11x learning rate improvements.

ianstephens

EfficientNetV2 really is something. As advertised for sure.

ianstephens

How strange. We fired up a model right away with no issues. However, I haven't tried to start any new models since I got that one started so it's possible it'll reappear in the future. I'll keep you updated. However, your suggestion of using MAE did fix the issue. But we've managed to get the model ...

ianstephens

Well, Mr. [mention]torzdf[/mention], we just updated to your latest commits (today) and managed to fire up a model straight away with recognizable previews appearing after 1,000 iterations. No more white boxes and corrupt models it seems. Thank you! P.s. what was the issue? It'd be interesting to le...

ianstephens

For the sake of an accurate test, I did it all again... Switched back to SSIM. Started a new model and got the white preview boxes straight away with no masks visible and they continued into the first save and going forward and never developed. Switched over to MAE. Started a new model and within 5...

ianstephens

Honestly, my suspicion would be learning rate too. Why this happens on TF 2,8, I'm not sure. Just to rule something out, does changing the loss to MAE also cause this issue? I think it's highly unlikely, but as part of the TF 2.8 update I had to change the SSIM loss implementation. I actually chang...

ianstephens

Yes, of course - completely delete the corrupt model folder and try again with different settings.

Strange thing is that we managed to get one model going after several attempts (albeit with a different dataset).

But now, no matter what we try, we can't get a model fired up.

Faceswap Forum

Search found 117 matches

Re: Potential VRAM Saving techniques

Re: Potential VRAM Saving techniques

Re: Potential VRAM Saving techniques

Re: Potential VRAM Saving techniques

Re: Obscure close - Fish Eye Training are there any Z axis training models

Re: Potential VRAM Saving techniques

Re: AutoClip: Any Feedback From Users?

Re: AutoClip: Any Feedback From Users?

Latest with "Learn Mask"

AutoClip: Any Feedback From Users?

Funny (but serious) Traning Question Regarding Merging Two Faces

Re: Bug: Updated to Latest faceswap version now crashing when starting training

Re: Bug: Updated to Latest faceswap version now crashing when starting training

Re: Bug: Updated to Latest faceswap version now crashing when starting training

Re: Bug: Updated to Latest faceswap version now crashing when starting training

Re: Bug: Updated to Latest faceswap version now crashing when starting training

Re: Bug: Updated to Latest faceswap version now crashing when starting training

Re: Bug: Updated to Latest faceswap version now crashing when starting training

Re: Bug: Updated to Latest faceswap version now crashing when starting training

Re: Bug: Updated to Latest faceswap version now crashing when starting training