Everything says to switch to "No Warp" eventually, but when I to, it looks clearly like the source face again.

If training is failing to start, and you are not receiving an error message telling you what to do, tell us about it here


Forum rules

Read the FAQs and search the forum before posting a new topic.

This forum is for reporting errors with the Training process. If you want to get tips, or better understand the Training process, then you should look in the Training Discussion forum.

Please mark any answers that fixed your problems so others can find the solutions.

Locked
User avatar
unkempt
Posts: 81
Joined: Wed Dec 28, 2022 2:09 pm
Has thanked: 1 time
Been thanked: 9 times

Everything says to switch to "No Warp" eventually, but when I to, it looks clearly like the source face again.

Post by unkempt »

So I ran training A face to B face (A to replace, B is the replacement). Using "warp to landmarks", it looks pretty solid after not too long, but the guides and tooltips all say that you should use "no warp" for final details. So I did that (and someone in a different thread explained that it's using BOTH Warp to Landmarks and NO Warp that's correct so I did that) and left it running overnight, but it's like three copies of the original face now.

To confirm, I did a conversion and it doesn't even look like a faceswap has happened for the most part. The face is also really blurry after 200k plus iterations. What am I doing wrong? This is with DFl-h128. Videos are simple in nature - think modeling shot with a single person like this: No other people, no complex backgrounds, some variation of face position, but mostly head-on.

by torzdf » Wed Feb 01, 2023 11:56 pm

Just my 2 cents.

Firstly this:

unkempt wrote: Tue Jan 31, 2023 2:16 pm

someone in a different thread explained that it's using BOTH Warp to Landmarks and NO Warp that's correct

is nonsense. Ignore it. (No warp means no warp. Doesn't matter if warp to landmarks is selected or not, it won't be used).

Some people have said that disabling warp leads to more identity leak. It is not an issue that has ever impacted me though, which makes me suspect that data quality is the most likely culprit.

Go to full post
User avatar
unkempt
Posts: 81
Joined: Wed Dec 28, 2022 2:09 pm
Has thanked: 1 time
Been thanked: 9 times

Re: Everything says to switch to "No Warp" eventually, but when I to, it looks clearly like the source face again.

Post by unkempt »

I just tested with a second video (same source face). Same result. As soon as I select "no warp", the result after 8 hours is that it looks like some kind of hybrid of the two faces instead of clearly the replacement face which is what it looked like before I clicked no warp. Near as I can tell, no warp consistently ruins training and should not be used.

User avatar
bryanlyon
Site Admin
Posts: 793
Joined: Fri Jul 12, 2019 12:49 am
Answers: 44
Location: San Francisco
Has thanked: 4 times
Been thanked: 215 times
Contact:

Re: Everything says to switch to "No Warp" eventually, but when I to, it looks clearly like the source face again.

Post by bryanlyon »

DFl-h128 is a particularly "blendy" model. It's known to not create particularly distinct A and B. It's very possible that model really requires warp to work reliably. Unfortunately that is not a model that I have much experience using and can't provide more precise advice but I can say a few things.

  • Make sure that your data has a lot of variety. I'd say that video has next to no variety at all, fine for a swap but you'd need variety to feed the model. Remember, variety is key. Having 20,000 images that all look close to the same is nearly the same as only having 1 image.

  • If your faces are extremely different warp might be necessary to get the model "close enough" that the AI can finish. In that case you might have to leave the warp enabled.

  • Final details happen only once the model is fully trained. "Blurry" faces is a sign that you're not ready for that yet, and you might be turning off the warp when the model doesn't know enough about the faces so it resorts to memorization/passing the face through from the source instead of learning how to create the face properly.

User avatar
torzdf
Posts: 2636
Joined: Fri Jul 12, 2019 12:53 am
Answers: 156
Has thanked: 128 times
Been thanked: 614 times

Re: Everything says to switch to "No Warp" eventually, but when I to, it looks clearly like the source face again.

Post by torzdf »

Just my 2 cents.

Firstly this:

unkempt wrote: Tue Jan 31, 2023 2:16 pm

someone in a different thread explained that it's using BOTH Warp to Landmarks and NO Warp that's correct

is nonsense. Ignore it. (No warp means no warp. Doesn't matter if warp to landmarks is selected or not, it won't be used).

Some people have said that disabling warp leads to more identity leak. It is not an issue that has ever impacted me though, which makes me suspect that data quality is the most likely culprit.

My word is final

User avatar
unkempt
Posts: 81
Joined: Wed Dec 28, 2022 2:09 pm
Has thanked: 1 time
Been thanked: 9 times

Re: Everything says to switch to "No Warp" eventually, but when I to, it looks clearly like the source face again.

Post by unkempt »

I may have answered my own question. The training thumb looks wonky, but I tried doing an actual conversion and it's... impressive. I saved the model folders before doing no warp and might compare the two, but it looks like I just need to trust the magic.

User avatar
unkempt
Posts: 81
Joined: Wed Dec 28, 2022 2:09 pm
Has thanked: 1 time
Been thanked: 9 times

Re: Everything says to switch to "No Warp" eventually, but when I to, it looks clearly like the source face again.

Post by unkempt »

torzdf wrote: Wed Feb 01, 2023 11:56 pm

Just my 2 cents.

Firstly this:

unkempt wrote: Tue Jan 31, 2023 2:16 pm

someone in a different thread explained that it's using BOTH Warp to Landmarks and NO Warp that's correct

is nonsense. Ignore it. (No warp means no warp. Doesn't matter if warp to landmarks is selected or not, it won't be used).

Some people have said that disabling warp leads to more identity leak. It is not an issue that has ever impacted me though, which makes me suspect that data quality is the most likely culprit.

Thanks for the clarification! I wrote in a following comment that the convert process SEEMS to work fine... I don't know why the training thumbs backpedal, but it seems to not affect the final result.

User avatar
hs1985
Posts: 8
Joined: Fri Jan 13, 2023 4:36 pm
Has thanked: 2 times

Re: Everything says to switch to "No Warp" eventually, but when I to, it looks clearly like the source face again.

Post by hs1985 »

unkempt wrote: Thu Feb 02, 2023 12:53 am

I may have answered my own question. The training thumb looks wonky, but I tried doing an actual conversion and it's... impressive. I saved the model folders before doing no warp and might compare the two, but it looks like I just need to trust the magic.

I ran into the same issue last week. I ran 900K iterations on DFl-H128 with "No warp" unchecked and another 200K iterations with "No warp" checked at the end. The thumbnails in the training tab looked good when the training finished, but the final converted video was a hybrid of the two faces. You said the final conversion was impressive in the above post. What did you do to improve it? Did you train with "No warp" unchecked during the whole training session or did you train a different model?

Last edited by hs1985 on Mon Feb 06, 2023 10:47 pm, edited 1 time in total.
User avatar
torzdf
Posts: 2636
Joined: Fri Jul 12, 2019 12:53 am
Answers: 156
Has thanked: 128 times
Been thanked: 614 times

Re: Everything says to switch to "No Warp" eventually, but when I to, it looks clearly like the source face again.

Post by torzdf »

I wouldn't advise training with no warp the whole time. It will most likely end badly.

My word is final

Locked