Is landmark accuracy important in convert?

[npbool]

I'm trying to convert a video. All face bounding boxes are correct. The landmarks are mostly ok but very inaccurate for side faces with obstructions. Suprisingly the BiSeNet-FP stil generates correct masks for these faces. Is it ok to leave landmarks uncorrected when masks are accurate in convert stage? Manully correcting landmarks for each frame are very time consuming.

[torzdf]

I'll repost something I placed in this wider thread re: landmarks:

So a few high-level tips.

1. Use bisenet-fp. It is by far and away the best mask in all circumstance. Particularly for handling obstructions
2. Don't worry about editing individual landmark points. That only exists because there was demand for it. The reality is that it is very rare that you will need that fine grained control
3. Don't fix masks until you have fixed your alignments. Any edits to the mask will get instantly erased (as will all NN generated masks for that face in that frame). Any landmarks edit, in any way, instantly invalidates masks
4. If you are not using the 'extended' or 'components' mask (you aren't) then you only need to worry about the landmarks in the 'core' area. (that is eyes, nose mouth, eyebrows) as these are the points that are actually used to align the image. The jawline is entirely unused, so you can probably skip faces where the jawline 'flies away'
5. Don't spend an inordinate amount of time fixing up training data. It should be generally correct, yes, but the model is seeing millions of faces, so a few incorrect alignments won't really hurt anything. It is easier to just delete bad training images than fix them all (unless data quality is at a premium)
6. Fixing for convert is far more important.
7. If you have a face which is particularly hard to get a 'lock' on the landmarks, first try switching between 'clahe' and 'hist' normalization in the manual tool. This can sometimes help. If that doesn't work, copy the alignments from the previous/next good face and use the "Extract box editor" (green box) to shift the good face's alignments to be an approximate match with the current face

I would also, in your use-case, run a test convert first and see how it comes out. Sometimes it will work out, other times it won't. Generally landmarks should be correct for converting. Often the image can look aligned, but actually it isn't, because the landmarks are actually all over the place, and the slight discrepancy of the face alignment leads to a poor convert. You won't know if you need to put the work in unless you run a test convert though.