Search found 9 matches
- Fri Jan 27, 2023 6:32 pm
- Forum: General Discussion
- Topic: Basic Dark Mode Theme
- Replies: 2
- Views: 6488
Basic Dark Mode Theme
I was also looking for a dark mode like this post I don't know Python/Tk very well but I was able to hack together a nowhere-near-perfect-but-satisfactory dark theme for myself, and I figured it might be of interest to other users. Screenshot of the theme: darkmode.jpg Installation is a bit tricky s...
- Fri Jan 20, 2023 6:05 pm
- Forum: Training Discussion
- Topic: Potential VRAM Saving techniques
- Replies: 34
- Views: 8883
Re: Potential VRAM Saving techniques
The really important and subtle part is if you have a good LR for a given model . What makes an LR good depends on a ton of things, including model structure, model size, loss function, and so on. In the context of this thread, we are dealing with running huge models that can only run at BS=1 or 2, ...
- Thu Jan 19, 2023 8:18 pm
- Forum: Training Discussion
- Topic: Potential VRAM Saving techniques
- Replies: 34
- Views: 8883
Re: Potential VRAM Saving techniques
MaxHunter If I understand that section of the paper correctly, they say that specifically using Adam optimizer learning rate scales proportionally to square root of batch size . This is very different from saying that the LR is square root of batch size . Essentially they are saying that if you hav...
- Thu Jan 19, 2023 7:51 pm
- Forum: Training Discussion
- Topic: LPIPS Alex vs Squeeze Surprising Behavior
- Replies: 4
- Views: 1138
Re: LPIPS Alex vs Squeeze Surprising Behavior
I see - so is there ever any advantage of using Squeeze over Alex since it uses more VRAM and runs as-fast-or-slower? It sounds like it's a smaller model in terms of stored size which is not particularly of concern for a training use-case. I've noticed VGG16: uses way more VRAM but gives significant...
- Wed Jan 18, 2023 12:10 am
- Forum: Training Discussion
- Topic: LPIPS Alex vs Squeeze Surprising Behavior
- Replies: 4
- Views: 1138
LPIPS Alex vs Squeeze Surprising Behavior
Both the documentation and the paper - "50x fewer parameters. ... 510x smaller than AlexNet" - describe Squeeze as lightweight compared to Alex. I do find that at the same batch rate, Squeeze is faster than Alex. I'd expect that Squeeze should also consume less VRAM than Alex or at worst t...
- Fri Jan 13, 2023 10:29 pm
- Forum: Training Discussion
- Topic: How do you reuse training/data from B source face for a new A Video/face?
- Replies: 6
- Views: 1818
Re: How do you reuse training/data from B source face for a new A Video/face?
how Faceswap recognize that in the new project i need the B face of the model (A/b long trained) for my new project C/B ? Look in the "load weights"/"freeze weights" section in the PhazeA options Which to load+freeze depends on your model. For an Fc(both) + decoder(A),decoder(B)...
- Fri Jan 13, 2023 8:55 pm
- Forum: Training Discussion
- Topic: Is MP always a VRAM saver / performance enhancer?
- Replies: 3
- Views: 832
Re: Is MP always a VRAM saver / performance enhancer?
Mixed precision will ALWAYS use less VRAM. It however, will not always be faster. That does depend on your GPU and system. Thank you for the clarification If you're on Windows a LOT of things can use your VRAM, even just opening the start menu takes some. In the end, it's very possible that your we...
- Fri Jan 13, 2023 6:37 am
- Forum: Training Discussion
- Topic: Is MP always a VRAM saver / performance enhancer?
- Replies: 3
- Views: 832
Is MP always a VRAM saver / performance enhancer?
There appears to be a complex relationship between model structure and how MP will behave. Some models run significantly faster or with higher batch sizes, but others don't really see any improvement at all. And in the worst case, I even built a model that somehow allowed me to run a batch size (4) ...
- Fri Jan 13, 2023 5:54 am
- Forum: Training Support
- Topic: SYM384 Model Preset yielding solid color blocks after a few thousand iterations
- Replies: 5
- Views: 1522
Re: SYM384 Model Preset yielding solid color blocks after a few thousand iterations
I also ran into this issue on SYM384 and other similarly high parameter-to-resolution-ratio (256-384px w/ 200k+ params) models After messing around a bunch, here are my observations on starting these models and avoiding Solid Color of Death (SCOD) ;) General params: enabling/disabling icnr init has ...