Best Model to Swap Different Face Shapes?

Want to understand the training process better? Got tips for which model to use and when? This is the place for you


Forum rules

Read the FAQs and search the forum before posting a new topic.

This forum is for discussing tips and understanding the process involved with Training a Faceswap model.

If you have found a bug are having issues with the Training process not working, then you should post in the Training Support forum.

Please mark any answers that fixed your problems so others can find the solutions.

Locked
User avatar
363LS2GTO
Posts: 30
Joined: Fri Jul 08, 2022 7:06 pm
Has thanked: 1 time
Been thanked: 4 times

Best Model to Swap Different Face Shapes?

Post by 363LS2GTO »

After performing several face swaps and even looking at some examples from the thread "choosing faces that match" it seems that the current models all have a pretty big problem.

The models I have seen do a good job swapping the eyes, nose, lips, and cheek bones but the chin, jawline, and distance from ear to ear do not change much if at all. This makes it pretty hard to perform a convincing swap without finding a nearly perfect match for the above listed areas.

I am not the first person to notice this 'problem' and I have read that models are being developed to address this issue.

I have read that lae based models do a better job at replicating face shape at the expense of face detail.

Other forks of deep fakes are using 'morphing' models to address this issue.

This reminds me of GPS technology in the 90's. It's pretty good nowhere near its potential.

What is the best current model faceswap.dev has to replicate the whole face? I know there is an option for head replication but this really only works in certain circumstances (mainly for people without hair).

Is this an issue that we are just going to have to wait for the technology to be allowed to evolve?

User avatar
torzdf
Posts: 2649
Joined: Fri Jul 12, 2019 12:53 am
Answers: 159
Has thanked: 128 times
Been thanked: 623 times

Re: Best Model to Swap Different Face Shapes?

Post by torzdf »

More often that not, this is a masking issue rather than a training issue. Face shape is difficult, for sure. If the A side has a wider face than the B side, then A's face will come through. If it's the other way around, it should be less problematic.

My word is final

User avatar
363LS2GTO
Posts: 30
Joined: Fri Jul 08, 2022 7:06 pm
Has thanked: 1 time
Been thanked: 4 times

Re: Best Model to Swap Different Face Shapes?

Post by 363LS2GTO »

I will give a swap a try using a smaller A face than B face. Using DFL-SAE on DF setting, this was a major problem for one of my swaps. The bleed through of A is really bad even though the faces were not too far apart. The A head was larger than the B head. The output looked like a head from Easter island.

Do you have any specific model recommendation to try or is there a way to configure Phaze A to address this issue?

I am still learning, but I check landmarks and masks in the manual tool for every face in my data set now. I also like getting results by outputting to photos as this is quick, very accurate, uses little space, and lets me know if a swap will work. If the swap is just not going to work, I don't see the point in making more than one short video clip (to go along with the photos) as this can be quite time consuming and for me this is just a fun hobby. At some point it starts to feel like work.

Using AI upscale I can get a clear image (literally) of how the swap is progressing at 20 hours compute time in villain.

I still see a lot of problem with the chin and jawline on everyone's examples. If you watch the gifs posted showing the results of multiple swaps with multiple models, while the 'interior' of the face changes the 'exterior' of the face does not change much if at all between all of the models. Maybe its related to the compute time necessary at the moment to do larger sections of the face. I have read that 'professionals' need to be fluent in multiple programs to achieve the results they are looking for and there is no one 'perfect' fire and forget program available.

For example, I have a $6 app that does a convincing job of altering the chin and jawline of a photo. Its just not useful for video.

I will keep trying different techniques as having something imperfect to learn with is better than nothing at all.

User avatar
torzdf
Posts: 2649
Joined: Fri Jul 12, 2019 12:53 am
Answers: 159
Has thanked: 128 times
Been thanked: 623 times

Re: Best Model to Swap Different Face Shapes?

Post by torzdf »

Chin is often because coverage is set too low, so the "swapped" chin ends up being eroded out, as it is so close to the edge of the swap area.

Also, some people just don't swap. I don't know why, but I have to abandon a few projects because I just could not get the face to swap.

My word is final

Locked