Search found 117 matches

by ianstephens
Thu Jan 18, 2024 2:38 pm
Forum: General Discussion
Topic: Building a GAN Model to provide "faces" from a limited dataset
Replies: 1
Views: 1801

Building a GAN Model to provide "faces" from a limited dataset

So I had a thought about the possibility of building and training a GAN model for generating more "faces" for a limited "face" dataset. The model would learn on the limited dataset and also the landmarks of the faces. Then, we would ask the model to provide different faces based ...
by ianstephens
Sun Oct 08, 2023 8:56 pm
Forum: Training Discussion
Topic: AutoClip Feedback from Personal AI models Outside FS
Replies: 1
Views: 3388

AutoClip Feedback from Personal AI models Outside FS

I wanted to reply to my previous autoclip post but it's "locked". However: Hi all, I wanted to share my two cents on using AutoClip outside faceswap. I have been working on other custom AI models not related to faceswap using Python/TF/Keras etc for my own personal projects. I have notice...
by ianstephens
Tue Aug 29, 2023 4:15 pm
Forum: Training Discussion
Topic: Yah new sutff, a vision transformer CLipV
Replies: 25
Views: 25859

Re: Yah new sutff, a vision transformer CLipV

I am also struggling to achieve more focus and clarity after a certain point. However losses are still dropping fairly nicely so I'm not too concerned yet. Hell, I guess I'm just used to efficientnetv2!

I will also be playing about with things and will report back!

by ianstephens
Sat Dec 24, 2022 8:04 pm
Forum: Training Discussion
Topic: [Guide] Introducing - Phaze-A
Replies: 89
Views: 164992

Re: [Guide] Introducing - Phaze-A

I, for the life of me, cannot get a SYM-384 model started. After the first model save (500its) the preview always turns out like this: Screenshot 2022-11-28 at 10.27.49.png I've tried lowering the learning rate as low as 1e-5 and still the same after the first model save (500its). I've also tried s...
by ianstephens
Tue Nov 29, 2022 12:01 pm
Forum: Training Discussion
Topic: Phaze-A, 2080Ti
Replies: 9
Views: 2226

Re: Phaze-A, 2080Ti

I tried Phaze-A with the 1024 presets and 2048 output res. its running fine with my 1080Ti at batchsize 70. It seems fantastic! I can already see start of a good model at 1750iter. Impossible. My thoughts too. Even with 24GB VRAM we only manage BS 2-3 with the DNY 1024 model. And seeing results fro...
by ianstephens
Tue Nov 29, 2022 11:54 am
Forum: Training Discussion
Topic: [Guide] Introducing - Phaze-A
Replies: 89
Views: 164992

Re: [Guide] Introducing - Phaze-A

I've currently shelved MP as I got sick of the NaNs. So for me, it will always be FP provided I am able with the current hardware @ianstephens I know you have a big rig, but out of curiosity what is your batch size and how long has it taken you to train a 512 on FP? MaxHunter : I've not played with...
by ianstephens
Mon Nov 28, 2022 10:32 am
Forum: Training Discussion
Topic: [Guide] Introducing - Phaze-A
Replies: 89
Views: 164992

Re: [Guide] Introducing - Phaze-A

I, for the life of me, cannot get a SYM-384 model started. After the first model save (500its) the preview always turns out like this: Screenshot 2022-11-28 at 10.27.49.png I've tried lowering the learning rate as low as 1e-5 and still the same after the first model save (500its). I've also tried st...
by ianstephens
Sat Nov 26, 2022 10:09 pm
Forum: Training Discussion
Topic: [Guide] Introducing - Phaze-A
Replies: 89
Views: 164992

Re: [Guide] Introducing - Phaze-A

If you can get away with not using mixed precision, do, as I had NaN issues with SYM-384 using MP. It resolves nice and quickly but NaNs earlier than I would have liked. Otherwise, I would say start with a fairly low lr. Maybe 3.5e-5, may 2e-5. Can't give more guidance beyond that sadly. Thanks for...
by ianstephens
Fri Nov 25, 2022 10:44 pm
Forum: Training Discussion
Topic: [Guide] Introducing - Phaze-A
Replies: 89
Views: 164992

Re: [Guide] Introducing - Phaze-A

Am just about to play with the sym384 model. The DNY 1024 is a great model but a very slow burner due to low batch size meaning crazily high iterations and time are needed (over a month+ to get clarity with our tested datasets) and I want to play with something that can run on efficientnetv2 as I've...
by ianstephens
Fri Sep 23, 2022 2:49 pm
Forum: Extract Support
Topic: Error Logged While Extracting - 32bit limitation?
Replies: 4
Views: 1163

Re: Error Logged While Extracting - 32bit limitation?

No problem, I am here to test things and report back :D You would require the video source file? I can provide it however it's ~ 75GB. The actual video resolution is 3840x3840 - perhaps that plays a part. With regards to (the now completed) extraction process - am I safe to assume everything has bee...
by ianstephens
Fri Sep 23, 2022 1:58 pm
Forum: Extract Support
Topic: Error Logged While Extracting - 32bit limitation?
Replies: 4
Views: 1163

Error Logged While Extracting - 32bit limitation?

We have stumbled upon a brand new error(s) I've not yet seen before in the extract process. 09/22/2022 20:40:29 INFO Initialized BiSeNet - Face Parsing (Mask) with batchsize of 8 Running pass 1 of 1: Extraction: 34%|████████████▊ | 48925/144519 [6:12:05<8:17:28, 3.20it/s] C:\Convert\plugins\extract\...
by ianstephens
Wed Aug 24, 2022 10:04 pm
Forum: Training Discussion
Topic: Potential VRAM Saving techniques
Replies: 34
Views: 8443

Re: Potential VRAM Saving techniques

I can't work out why I'm now able to run BS5 all-on-GPU with the existing 1024 model I have been working with. It's stable too and no OOM with certain batches and having to drop down an integer.

Could it be to do with, "Process data at model input/output size rather than saved image size"?

by ianstephens
Tue Aug 23, 2022 5:30 pm
Forum: Training Discussion
Topic: Potential VRAM Saving techniques
Replies: 34
Views: 8443

Re: Potential VRAM Saving techniques

I've just added the graph to show GPU utilization after I was sure the entire dataset had been cycled through: https://forum.faceswap.dev/viewtopic.php?p=7441#p7441 As you can see, GPU is utilized much more once this is cached/processed or whatever goes on behind the scenes. However, of course, it's...
by ianstephens
Tue Aug 23, 2022 11:59 am
Forum: Training Discussion
Topic: Potential VRAM Saving techniques
Replies: 34
Views: 8443

Re: Potential VRAM Saving techniques

Fixes an issue where masks would sometimes be slightly misaligned

That's fantastic. I've hit this one a few times. The mask is accurate but not aligned correctly to the image.

by ianstephens
Tue Aug 23, 2022 11:47 am
Forum: Training Discussion
Topic: Potential VRAM Saving techniques
Replies: 34
Views: 8443

Re: Potential VRAM Saving techniques

Utilization graph for DNY 1024 (1536px images) BS5 default strategy (all on GPU) before "caching/processing" has cycled through the entire dataset: Screen Shot 2022-08-23 at 12.44.00.png After "caching/processing" has cycled through the entire dataset: Screen Shot 2022-08-23 at 1...
by ianstephens
Tue Aug 23, 2022 7:39 am
Forum: Training Discussion
Topic: Potential VRAM Saving techniques
Replies: 34
Views: 8443

Re: Potential VRAM Saving techniques

I have updated the above post with results after updating:
viewtopic.php?p=7422#p7422

As you can see, there is a definite improvement.

I will now move on to testing without central storage enabled.

by ianstephens
Mon Aug 22, 2022 9:43 pm
Forum: Training Discussion
Topic: Potential VRAM Saving techniques
Replies: 34
Views: 8443

Re: Potential VRAM Saving techniques

That's interesting that any of the updates wouldn't affect VRAM usage and batch size - don't know what happened there. GPU utilization (on the same settings as the last screenshot) certainly improved albeit not massively but any little helps: Screen Shot 2022-08-22 at 22.38.47.png The lows and peaks...
by ianstephens
Mon Aug 22, 2022 5:51 pm
Forum: Training Discussion
Topic: Potential VRAM Saving techniques
Replies: 34
Views: 8443

Re: Potential VRAM Saving techniques

Update: I want to run at least 20-30k iterations as I have with the previous data above. The reason being - from what I can tell - it appears onece the new caching/processing you mentioned seems to have cycled through the entire image set, the time between iterations started becoming noticeably fas...
by ianstephens
Mon Aug 22, 2022 2:04 pm
Forum: Training Discussion
Topic: Potential VRAM Saving techniques
Replies: 34
Views: 8443

Re: Potential VRAM Saving techniques

I thought an easy way to gauge would be to run the exact same settings with the only change being updating to the current codebase with the optimizations applied. Then we can look at EGs/s. Running DNY 1024 model with 1536px images. Side A has 8000 images and side B has 5000. We are running batch si...
by ianstephens
Mon Aug 22, 2022 11:58 am
Forum: Training Discussion
Topic: Potential VRAM Saving techniques
Replies: 34
Views: 8443

Re: Potential VRAM Saving techniques

Let me run some tests and grab some data. Leave it with me :D