256 High Batch Size Progress

Want to understand the training process better? Got tips for which model to use and when? This is the place for you


Forum rules

Read the FAQs and search the forum before posting a new topic.

This forum is for discussing tips and understanding the process involved with Training a Faceswap model.

If you have found a bug are having issues with the Training process not working, then you should post in the Training Support forum.

Please mark any answers that fixed your problems so others can find the solutions.

Locked
User avatar
EXHYST
Posts: 3
Joined: Sat Nov 09, 2019 9:51 am

256 High Batch Size Progress

Post by EXHYST »

Hey

I have been trainning at the max batch size of 256 for more than 1 day.

Did not have any crashes.

But I read that " there are slight quality degradations at very large batch sizes".. So should I reduce it? And reduce it to what?

Image

After about 60hours and 90k iterations my faceswap looks like this:

If I continue will it improve much more?

I dont see much progress in the last 10 hours of trainning.

Thanks

User avatar
torzdf
Posts: 2665
Joined: Fri Jul 12, 2019 12:53 am
Answers: 159
Has thanked: 131 times
Been thanked: 625 times

Re: 256 High Batch Size Progress

Post by torzdf »

If you haven't already, you should read the training guide, as it talks about selecting the batch size
viewtopic.php?f=6&t=146

The Best Practice tips also offer some pointers for eeking out more quality:
viewtopic.php?f=6&t=74

My word is final

User avatar
bryanlyon
Site Admin
Posts: 793
Joined: Fri Jul 12, 2019 12:49 am
Answers: 44
Location: San Francisco
Has thanked: 4 times
Been thanked: 218 times
Contact:

Re: 256 High Batch Size Progress

Post by bryanlyon »

More specifically, you can feel free to change the batch size as you train. Lowering the BS to 64 near the end of training makes sense if you want to see the differences of a smaller BS. It wont hurt to try different values if you want to see how it affects your results.

Locked