What does it mean when training starts wonky?

If training is failing to start, and you are not receiving an error message telling you what to do, tell us about it here


Forum rules

Read the FAQs and search the forum before posting a new topic.

This forum is for reporting errors with the Training process. If you want to get tips, or better understand the Training process, then you should look in the Training Discussion forum.

Please mark any answers that fixed your problems so others can find the solutions.

Locked
User avatar
cagonzon
Posts: 4
Joined: Sun Jan 02, 2022 4:38 am
Has thanked: 1 time
Been thanked: 1 time

What does it mean when training starts wonky?

Post by cagonzon »

Hello! (I posted this erroneously in Training Discussion, first, apologies)

The first model I trained went pretty well..

The second model, doing the same procedure from scratch, not so much. From the start it's been showing this (image) for the A>A and A>B Image

the only difference in procedure this time compared to the first is that I have about 30k extracted images as a source for A, while for the first model I have about 3k.

This second model training is 10k iterations in and it hasn't changed. The masks are there in the extracted faces, but the images are these blanks. Also, face B is the same extracted sources I used for the first model. I wanted to do this second model from scratch.

Any thoughts? Corrupted from the start? This was from the beginning, so it's not like the model started warping and turning into a solid color (like in the FAQ) it was always wrong.

Thanks for any input.

Attachments
Screenshot 2022-01-07 143849.jpg
Screenshot 2022-01-07 143849.jpg (28.5 KiB) Viewed 1076 times
User avatar
cagonzon
Posts: 4
Joined: Sun Jan 02, 2022 4:38 am
Has thanked: 1 time
Been thanked: 1 time

Re: What does it mean when training starts wonky?

Post by cagonzon »

I took it back to the start. New Face A and B, extracted, cleaned up, began training, and the same thing happens. Can't figure out what I'm doing wrong.

Attachments
Screenshot 2022-01-10 162823.jpg
Screenshot 2022-01-10 162823.jpg (34.47 KiB) Viewed 1058 times
User avatar
abigflea
Posts: 182
Joined: Sat Feb 22, 2020 10:59 pm
Answers: 2
Has thanked: 20 times
Been thanked: 62 times

Re: What does it mean when training starts wonky?

Post by abigflea »

This is Phaze-A , correct?

:o I dunno what I'm doing :shock:
2X RTX 3090 : RTX 3080 : RTX: 2060 : 2x RTX 2080 Super : Ghetto 1060

User avatar
torzdf
Posts: 2651
Joined: Fri Jul 12, 2019 12:53 am
Answers: 159
Has thanked: 129 times
Been thanked: 622 times

Re: What does it mean when training starts wonky?

Post by torzdf »

Yeah, really depends on the model you're using. If it's failing to start training though, you should try lowering the learning rate.

My word is final

Locked