"White dot" and "Profile"... any suggestion ?

Want to understand the training process better? Got tips for which model to use and when? This is the place for you


Forum rules

Read the FAQs and search the forum before posting a new topic.

This forum is for discussing tips and understanding the process involved with Training a Faceswap model.

If you have found a bug are having issues with the Training process not working, then you should post in the Training Support forum.

Please mark any answers that fixed your problems so others can find the solutions.

Locked
User avatar
Grassone
Posts: 45
Joined: Sun Apr 19, 2020 7:32 pm
Has thanked: 6 times
Been thanked: 2 times

"White dot" and "Profile"... any suggestion ?

Post by Grassone »

Hello.

My results are improving (even on this toy pee-see): just a couple of question:
1)
"White dot" on the iris
Any advice on how to make it work?

dot.jpg
dot.jpg (43.23 KiB) Viewed 1682 times

Do I have to use an high end trainer (I am using the DFL-H128) or is it a matter of iterations ? (i have done up to 1.300.000 but it doesn't appear anywhere)

2)
Profile faces:
I have to do a lot of manual alignment here:
Position and width of the eyes landmark are often too small and misplaced. The main effect is that the eyeball is usually blurry and almost with no white part at all.
Any advice ? I have collected A LOT of profile images, tried to correct the position and the dimension of the alignment as much as I can... but with very little luck. Occasionally I notice also that the eye landmarks (which are supposed to be an ellpse or similar) become a "butterfly shape" This especially on the far away eye, when the face is 3/4 profile.

Any advice will be appreciated.

User avatar
bryanlyon
Site Admin
Posts: 793
Joined: Fri Jul 12, 2019 12:49 am
Answers: 44
Location: San Francisco
Has thanked: 4 times
Been thanked: 218 times
Contact:

Re: "White dot" and "Profile"... any suggestion ?

Post by bryanlyon »

For the pupil shine:

If it's too small of a detail, it'll probably never show up. The Neural Network has a limited amount of internal space (technically called latent space) to dedicate to any details, and I'm sorry, but a few pixels in the middle of the eye are just unlikely to ever merit that space. Basically, everything that it HAS learned is more important and so keeps the pupil shine from coming through. This will apply to any and all models will suffer from similar issues (though DFL-H128 is particularly weak to this problem due to an even more constrained latent space).

For profiles:

In general profiles aren't going to come through, this isn't just a matter of alignments but of data quantity. I've seen people who have profile ONLY models that can get acceptable results given enough effort, but then you'll have to train two models and carefully convert each. Don't worry TOO much about the actual landmarks, they're not actually fed into the network and are really only used for generating the facesets and masks. If the face is centered in the set don't worry so much about the exact positioning of the landmarks.

Locked