Does anyone know if the loss-function's eye & mouth multiplier come at the cost of slowing other facial features down?

Want to understand the training process better? Got tips for which model to use and when? This is the place for you


Forum rules

Read the FAQs and search the forum before posting a new topic.

This forum is for discussing tips and understanding the process involved with Training a Faceswap model.

If you have found a bug are having issues with the Training process not working, then you should post in the Training Support forum.

Please mark any answers that fixed your problems so others can find the solutions.

Locked
User avatar
cosmico
Posts: 95
Joined: Sat Jan 18, 2020 6:32 pm
Has thanked: 13 times
Been thanked: 35 times

Does anyone know if the loss-function's eye & mouth multiplier come at the cost of slowing other facial features down?

Post by cosmico »

You don't have to have the perfect answer, I'm looking for any insight from the community here,

The eye and mouth multiplier increase the speed at which the eye and mouth are trained, which is pretty useful as the eyes and mouth are usually easy ways to tell that the model hasn't trained enough. ...But are there any consequences to using this?

Would a high eye and mouth multiplier loss function cause the nose, eyebrows, and other facial features to receive less attention as they try to improve with each iteration? Possibly because the eye and mouth are what the loss function is focused on? So in other words, once I'm happy with the eyes and mouth, should I put the multipliers back on default???

And since I'm asking questions on the subject....

What exactly constitutes "the eye" for the eye multiplier? Is it just the pupils and iris? The entire eye shape and eyelashes? The eyebrows too? I can tell that the mouth multiplier appears to affect the lips, teeth, tongue and inside of mouth.

Is there any reason why I shouldn't start a new model, crank both eye and mouth multiplier up to 40 and just keep it there until I've decided the model has been properly trained? -I mean, nothing in the descriptions say that it slows other facial structures down or can lead to model crash

And this question is mainly for the Devs: Assuming there's no harm or foul play in using these excessively, can we get a nose multiplier, a eyebrow multiplier, and a cheek multiplier? -L2 reg term function sounds like its a multiplier for the entire face, like what I'm suggesting, but its for SSIM and GMSD only ....and the description tells noobies to beware.

User avatar
bryanlyon
Site Admin
Posts: 793
Joined: Fri Jul 12, 2019 12:49 am
Answers: 44
Location: San Francisco
Has thanked: 4 times
Been thanked: 218 times
Contact:

Re: Does anyone know if the loss-function's eye & mouth multiplier come at the cost of slowing other facial features dow

Post by bryanlyon »

It multiplies the loss score for those areas. So yes, it does come at a cost of other areas of the face. The numbers that are set come from academic papers on the subject, but some basic experimentation has shown that too high of weighting definitely negatively effects results.

User avatar
cosmico
Posts: 95
Joined: Sat Jan 18, 2020 6:32 pm
Has thanked: 13 times
Been thanked: 35 times

Re: Does anyone know if the loss-function's eye & mouth multiplier come at the cost of slowing other facial features dow

Post by cosmico »

Even though it's not written I've sortve been operating under this assumption. Do you guys have any interest in adding additional loss multipliers like to the cheeks, nose, and eyebrows?

User avatar
bryanlyon
Site Admin
Posts: 793
Joined: Fri Jul 12, 2019 12:49 am
Answers: 44
Location: San Francisco
Has thanked: 4 times
Been thanked: 218 times
Contact:

Re: Does anyone know if the loss-function's eye & mouth multiplier come at the cost of slowing other facial features dow

Post by bryanlyon »

No, cheeks are not something that really get swapped as there isn't much data and wouldn't benefit from a multiplier. Nose has been a consideration, but we've never found it falling behind so didn't bother.

Locked