Search found 9 matches

by couleurs
Fri Jan 27, 2023 6:32 pm
Forum: General Discussion
Topic: Basic Dark Mode Theme
Replies: 2
Views: 6488

Basic Dark Mode Theme

I was also looking for a dark mode like this post I don't know Python/Tk very well but I was able to hack together a nowhere-near-perfect-but-satisfactory dark theme for myself, and I figured it might be of interest to other users. Screenshot of the theme: darkmode.jpg Installation is a bit tricky s...
by couleurs
Fri Jan 20, 2023 6:05 pm
Forum: Training Discussion
Topic: Potential VRAM Saving techniques
Replies: 34
Views: 8883

Re: Potential VRAM Saving techniques

The really important and subtle part is if you have a good LR for a given model . What makes an LR good depends on a ton of things, including model structure, model size, loss function, and so on. In the context of this thread, we are dealing with running huge models that can only run at BS=1 or 2, ...
by couleurs
Thu Jan 19, 2023 8:18 pm
Forum: Training Discussion
Topic: Potential VRAM Saving techniques
Replies: 34
Views: 8883

Re: Potential VRAM Saving techniques

MaxHunter If I understand that section of the paper correctly, they say that specifically using Adam optimizer learning rate scales proportionally to square root of batch size . This is very different from saying that the LR is square root of batch size . Essentially they are saying that if you hav...
by couleurs
Thu Jan 19, 2023 7:51 pm
Forum: Training Discussion
Topic: LPIPS Alex vs Squeeze Surprising Behavior
Replies: 4
Views: 1138

Re: LPIPS Alex vs Squeeze Surprising Behavior

I see - so is there ever any advantage of using Squeeze over Alex since it uses more VRAM and runs as-fast-or-slower? It sounds like it's a smaller model in terms of stored size which is not particularly of concern for a training use-case. I've noticed VGG16: uses way more VRAM but gives significant...
by couleurs
Wed Jan 18, 2023 12:10 am
Forum: Training Discussion
Topic: LPIPS Alex vs Squeeze Surprising Behavior
Replies: 4
Views: 1138

LPIPS Alex vs Squeeze Surprising Behavior

Both the documentation and the paper - "50x fewer parameters. ... 510x smaller than AlexNet" - describe Squeeze as lightweight compared to Alex. I do find that at the same batch rate, Squeeze is faster than Alex. I'd expect that Squeeze should also consume less VRAM than Alex or at worst t...
by couleurs
Fri Jan 13, 2023 10:29 pm
Forum: Training Discussion
Topic: How do you reuse training/data from B source face for a new A Video/face?
Replies: 6
Views: 1818

Re: How do you reuse training/data from B source face for a new A Video/face?

how Faceswap recognize that in the new project i need the B face of the model (A/b long trained) for my new project C/B ? Look in the "load weights"/"freeze weights" section in the PhazeA options Which to load+freeze depends on your model. For an Fc(both) + decoder(A),decoder(B)...
by couleurs
Fri Jan 13, 2023 8:55 pm
Forum: Training Discussion
Topic: Is MP always a VRAM saver / performance enhancer?
Replies: 3
Views: 832

Re: Is MP always a VRAM saver / performance enhancer?

Mixed precision will ALWAYS use less VRAM. It however, will not always be faster. That does depend on your GPU and system. Thank you for the clarification If you're on Windows a LOT of things can use your VRAM, even just opening the start menu takes some. In the end, it's very possible that your we...
by couleurs
Fri Jan 13, 2023 6:37 am
Forum: Training Discussion
Topic: Is MP always a VRAM saver / performance enhancer?
Replies: 3
Views: 832

Is MP always a VRAM saver / performance enhancer?

There appears to be a complex relationship between model structure and how MP will behave. Some models run significantly faster or with higher batch sizes, but others don't really see any improvement at all. And in the worst case, I even built a model that somehow allowed me to run a batch size (4) ...
by couleurs
Fri Jan 13, 2023 5:54 am
Forum: Training Support
Topic: SYM384 Model Preset yielding solid color blocks after a few thousand iterations
Replies: 5
Views: 1522

Re: SYM384 Model Preset yielding solid color blocks after a few thousand iterations

I also ran into this issue on SYM384 and other similarly high parameter-to-resolution-ratio (256-384px w/ 200k+ params) models After messing around a bunch, here are my observations on starting these models and avoiding Solid Color of Death (SCOD) ;) General params: enabling/disabling icnr init has ...