Page 1 of 1
Training only B side
Posted: Mon Sep 18, 2023 1:57 pm
by jode
Maybe stupid question but is it possible to train only B side? I often use same B side to different A sides. Hoping I could make B side "ready" for next project and then just load weights to make training faster. Is it possible with Phaze-A trainer?
Re: Training only B side
Posted: Tue Sep 19, 2023 2:03 am
by MaxHunter
The short answer is, no.
What you can do, if you use Phaze A, is freeze the B side, and reuse the weights. As you continually re-use the weights in theory it should take less time to train.
The model needs both sides to build properly.
Re: Training only B side
Posted: Wed Sep 20, 2023 10:04 am
by torzdf
Absolutely you can do this, to an extent. It will save time for the next training, but not completely eliminate.
You would
- create a new model in Phaze-A, same structure as the initial model.
- In the "load weights" section, load in weights for every part of the model that are "shared", "both" or "B" only
- Freeze the same weights as you selected for "load weights"
- In the main training window, "Load Weights" from the model that has the B data you want to keep
- Start training
The A side should start to catch up pretty quickly with the B-Side (you can train higher batch-size too, as less of the model is training).
Once you are happy that the A has got as far as it is going to get, unfreeze the whole model to let the rest of it adapt to the new data.