Gape between A and B on training graph

If training is failing to start, and you are not receiving an error message telling you what to do, tell us about it here


Forum rules

Read the FAQs and search the forum before posting a new topic.

This forum is for reporting errors with the Training process. If you want to get tips, or better understand the Training process, then you should look in the Training Discussion forum.

Please mark any answers that fixed your problems so others can find the solutions.

Locked
User avatar
lighting
Posts: 11
Joined: Thu Mar 02, 2023 6:54 pm
Has thanked: 1 time
Been thanked: 1 time

Gape between A and B on training graph

Post by lighting »

Hi to everyone.
I use Dfaker model with output 128, and now hit 100k iteration. Each training set contain 1019 faces, batch size set to 32. Everything goes well (by my opinion), but i have strange gap between Face A and Face B on graph. In the beginning they go approx. equally, but then with two jumps they separated on 0,01.
Image
This two jumps have reasons:
1) I set checkbox "Warp to landmarks" which forgot to set at start.
2) Change eye(from 1 to 3 ) and mouth(from 1 to 2) multiplier.

Does such behavior is normal or i missed out something? This gape will decline later on training? Should i worry about it or it does not really matter?
For now it take almost month of non stop training, not pretty fast on my taste :)

Last edited by lighting on Fri May 05, 2023 8:16 am, edited 2 times in total.
User avatar
torzdf
Posts: 2672
Joined: Fri Jul 12, 2019 12:53 am
Answers: 159
Has thanked: 131 times
Been thanked: 625 times

Re: Gape between A and B on training graph

Post by torzdf »

It's normal. You don't need to worry about it.

My word is final

User avatar
lighting
Posts: 11
Joined: Thu Mar 02, 2023 6:54 pm
Has thanked: 1 time
Been thanked: 1 time

Re: Gape between A and B on training graph

Post by lighting »

Thanks for answer. "Don't worry - be happy"

Locked