(Suggestion) Implement the ability to zoom farther in on the mesh.

Want to know about the Faceswap's Face Extraction process? Got tips, ideas or just want to learn about how it all works? Then this is the place for you


Forum rules

Read the FAQs and search the forum before posting a new topic.

This forum is for discussing tips and understanding the process involved for Extracting and preparing face sets for training a model in Faceswap.

If you have found a bug are having issues with the Extraction process not working, then you should post in the Extract Support forum.

Please mark any answers that fixed your problems so others can find the solutions.

Locked
User avatar
phillyphan2965
Posts: 1
Joined: Sun Jan 09, 2022 5:02 pm

(Suggestion) Implement the ability to zoom farther in on the mesh.

Post by phillyphan2965 »

As I go through my first time manually correcting faces, I find myself wasting a lot of time trying to pick exactly which landmark I need to move, especially when the face is very misaligned. Many times, I would accidentally select a landmark that should be used for the mouth when my goal is to edit the mesh of the jaw (this could use an undo button as well). After I realized the landmarks were numbered for a reason, I was able to be a little more precise but it takes a while to get the one you are after, when the face is a jumbled mess.

Adding the ability to zoom farther in when editing your landmarks would help greatly to quickly edit misaligned faces. In addition, differentiating somehow between the different meshes (ie mouth, nose, eyes, etc.) would also help so that if you only need to edit a certain aspect, you know just by looking, which landmark needs to be moved.

Finally, is there somewhere I can find a template for what "constitutes" different landmark features? The mouth is often the hardest to edit because there are many points that I am not sure exactly where on the mouth each is supposed to correspond to. It seems that the "mouth" mesh has a set number of landmarks that does not change, but a guide to where each point should go would be very helpful. Anywhere that I can find this?

User avatar
abigflea
Posts: 182
Joined: Sat Feb 22, 2020 10:59 pm
Answers: 2
Has thanked: 20 times
Been thanked: 62 times

Re: (Suggestion) Implement the ability to zoom farther in on the mesh.

Post by abigflea »

being super accurate with landmarks in training data will only make a minor difference.
I used to spend like 80+ hours "fixing" landmarks for 12K photos for training data till it was perfect.
In practice, close enough is good for training. Bryanlyon & Torzdf only had to tell me like 10 times

In general why this is:
The landmarks are used to generate the mask.
The mask is what is used to train with. This tells the network what is important to learn, not the alignments exactly.

I will go though and roughly fix alignments that are way off, of pics/data I want to keep, then tidy up the mask that generates. I limit the crazy exacting I used to do for training data.

On the "to convert" data/video, I really try to get the alignments right and the mask reasonably tight.... so I would focus your energy there more so :-)

Here you can read and see landmarks if you still want to be a bit nit picky.
https://www.pyimagesearch.com/2017/04/0 ... cv-python/

:o I dunno what I'm doing :shock:
2X RTX 3090 : RTX 3080 : RTX: 2060 : 2x RTX 2080 Super : Ghetto 1060

User avatar
torzdf
Posts: 2651
Joined: Fri Jul 12, 2019 12:53 am
Answers: 159
Has thanked: 129 times
Been thanked: 622 times

Re: (Suggestion) Implement the ability to zoom farther in on the mesh.

Post by torzdf »

phillyphan2965 wrote: Sun Jan 09, 2022 8:40 pm

Finally, is there somewhere I can find a template for what "constitutes" different landmark features? The mouth is often the hardest to edit because there are many points that I am not sure exactly where on the mouth each is supposed to correspond to. It seems that the "mouth" mesh has a set number of landmarks that does not change, but a guide to where each point should go would be very helpful. Anywhere that I can find this?

Zoom would be useful, but the work required vs reward to implement is not really worth it given the long list of priorities I have and the limited time.

There is an image in the extract guide re: landmarks:

Image

Pro-tip: If you change the aligner to "cv2-dnn" and just click on the face, it will basically lay the landmarks out correctly even if it can't find a face. You can then adjust from there.

My word is final

Locked