training with profile faces

Want to understand the training process better? Got tips for which model to use and when? This is the place for you


Forum rules

Read the FAQs and search the forum before posting a new topic.

This forum is for discussing tips and understanding the process involved with Training a Faceswap model.

If you have found a bug are having issues with the Training process not working, then you should post in the Training Support forum.

Please mark any answers that fixed your problems so others can find the solutions.

Locked
User avatar
deepfakesweb
Posts: 6
Joined: Mon Mar 15, 2021 6:25 pm
Has thanked: 3 times
Been thanked: 1 time

training with profile faces

Post by deepfakesweb »

Hey,

I had a question about training profile faces and their alignments.

https://ibb.co/n39dyC4

Should I be manually deleting the whole eye landmarks for his right eye? Or does the AI understand what's going on?

Appreciate feedback.

User avatar
torzdf
Posts: 2651
Joined: Fri Jul 12, 2019 12:53 am
Answers: 159
Has thanked: 129 times
Been thanked: 622 times

Re: training with profile faces

Post by torzdf »

You can't delete the "right eye". All 68 Landmarks need to be present. Therefore for profile shots the hidden landmarks will tend to "stack" onto the side of the face.

As long as the face image looks aligned, this is fine.

My word is final

User avatar
deepfakesweb
Posts: 6
Joined: Mon Mar 15, 2021 6:25 pm
Has thanked: 3 times
Been thanked: 1 time

Re: training with profile faces

Post by deepfakesweb »

Thanks torzdf. Could you take another look at this preview snapshot and perhaps give me your best guess as to whats going on?

https://ibb.co/RhjWMjt

I'm fairly happy with 4/5 previews, its just that one in the middle that is not turning out right. I haven't disabled "No Warp" yet, as I'm waiting for the previews to turn out all satisfactory before disabling.

The source is a 1080p video (torrent, not youtube), training on an RTX2080, realface trainer. 225k iterations.

The things that stand out to me are that;
1) it's a profile face, most previews of the profile face seem to be turning out this way.
2) the lighting is a bit dark, but not sure it's TOO dark?

Any possible solutions are appreciaited.

User avatar
bryanlyon
Site Admin
Posts: 793
Joined: Fri Jul 12, 2019 12:49 am
Answers: 44
Location: San Francisco
Has thanked: 4 times
Been thanked: 218 times
Contact:

Re: training with profile faces

Post by bryanlyon »

Profiles are always tough and often require postprocessing to get the best result. The problem is that you're blurring the edges and the edges in that case has a new nose sticking out past the old one. That's getting blurred with the background. There is no automatic solution for something like that. Instead you'd want to composite it manually so you can handle that (literal) edge case.

User avatar
deepfakesweb
Posts: 6
Joined: Mon Mar 15, 2021 6:25 pm
Has thanked: 3 times
Been thanked: 1 time

Re: training with profile faces

Post by deepfakesweb »

thanks for your reply bryan, when you say composite it manually, are you talking about fixing the alignments of the nose? Or doing something to make the source video more clear?

If it's the former, does something like this do the job or am I a complete moron?

original alignment: https://ibb.co/mJnwtNK

simply changed landmark "31" to the edge of his nose.

correction alignment: https://ibb.co/NrykgBs

appreciate your help.

User avatar
bryanlyon
Site Admin
Posts: 793
Joined: Fri Jul 12, 2019 12:49 am
Answers: 44
Location: San Francisco
Has thanked: 4 times
Been thanked: 218 times
Contact:

Re: training with profile faces

Post by bryanlyon »

No compositing is the process of combining two images into one. Compositing in my example would mean using video editting software like After Effects to combine the images. You would probably need to do it frame-by-frame in order to get the best results.

Locked