Can setting the Learning Rate to lower values help in getting the details better?

Want to understand the training process better? Got tips for which model to use and when? This is the place for you


Forum rules

Read the FAQs and search the forum before posting a new topic.

This forum is for discussing tips and understanding the process involved with Training a Faceswap model.

If you have found a bug are having issues with the Training process not working, then you should post in the Training Support forum.

Please mark any answers that fixed your problems so others can find the solutions.

Locked
User avatar
ugramund
Posts: 27
Joined: Sat Feb 20, 2021 3:56 am
Location: India
Has thanked: 25 times
Been thanked: 4 times

Can setting the Learning Rate to lower values help in getting the details better?

Post by ugramund »

After training for a good amount of time (with warp on), if we reduce the learning rate to very low values like 1e or 2e on a small batch of 4 or 8 (with warp off) & train for some time & again set the learning rate to normal, will it help to get the details of the facial feature better?

In love with the the RealFace model.

User avatar
bryanlyon
Site Admin
Posts: 793
Joined: Fri Jul 12, 2019 12:49 am
Answers: 44
Location: San Francisco
Has thanked: 4 times
Been thanked: 218 times
Contact:

Re: Can setting the Learning Rate to lower values help in getting the details better?

Post by bryanlyon »

Potentially, but it'll also take longer and could lead to overtraining if taken too low.

Locked