Halo effect around face on side B causes side A to have halo effect and not cover full face.

Want to understand the training process better? Got tips for which model to use and when? This is the place for you


Forum rules

Read the FAQs and search the forum before posting a new topic.

This forum is for discussing tips and understanding the process involved with Training a Faceswap model.

If you have found a bug are having issues with the Training process not working, then you should post in the Training Support forum.

Please mark any answers that fixed your problems so others can find the solutions.

Locked
User avatar
MR.Tibbs
Posts: 2
Joined: Mon Dec 06, 2021 7:12 pm
Has thanked: 2 times

Halo effect around face on side B causes side A to have halo effect and not cover full face.

Post by MR.Tibbs »

The mask and alignment for both sides are almost perfect. When using the manual tool to look at the frames, the mask and alignments cover up the face correctly. But as I’m training, side b doesn’t cover the full face on side A causes a halo effect when converting. I tried changing a bunch of the convert settings and it doesn’t work. e.g. Box blend, Mask blend, color, etc. Only changing “erosion” has very little effect but it takes too much of the face away and doesn’t completely overlay the face you are trying to replace. I have tried with multiple sources and it does the same thing. I don’t notice a difference when training longer. Mask type and color adjustment doesn’t make any difference either. The training set for both sides is 1024x1024. It wouldn't be too bad if I could blend the halo effect into the face, but nothing is doing that. Is there anything I’m missing or need to change?
I am using Phaze-A. RTX3060

Image
Image
Image
Image
Image

User avatar
bryanlyon
Site Admin
Posts: 793
Joined: Fri Jul 12, 2019 12:49 am
Answers: 44
Location: San Francisco
Has thanked: 4 times
Been thanked: 218 times
Contact:

Re: Halo effect around face on side B causes side A to have halo effect and not cover full face.

Post by bryanlyon »

You wont find a fix for this in the model settings.

In the case of vastly different face shape you're going to need to do manual compositing to get good results. To do that, change the "writer" to OpenCV or Pillow and enable "draw Transparent" to get just the model output. You can then manually blend it back into the original image.

User avatar
MR.Tibbs
Posts: 2
Joined: Mon Dec 06, 2021 7:12 pm
Has thanked: 2 times

Re: Halo effect around face on side B causes side A to have halo effect and not cover full face.

Post by MR.Tibbs »

bryanlyon wrote: Mon Jan 03, 2022 4:31 pm

You wont find a fix for this in the model settings.

In the case of vastly different face shape you're going to need to do manual compositing to get good results. To do that, change the "writer" to OpenCV or Pillow and enable "draw Transparent" to get just the model output. You can then manually blend it back into the original image.

Thank you for the reply. I changed the writer to pillow and enable "draw Transparent". I have the output. But I don't know what you mean by "manually blend it back into the original image". What do you mean and/or is there a guide to that?

Thank you

User avatar
torzdf
Posts: 2651
Joined: Fri Jul 12, 2019 12:53 am
Answers: 159
Has thanked: 129 times
Been thanked: 622 times

Re: Halo effect around face on side B causes side A to have halo effect and not cover full face.

Post by torzdf »

He means using an external tool (e,g. After Effects/Premiere/many other video editing suites) to load in the frames and composite there. I have no experience of this, so would need to leave to others to advise.

My word is final

Locked