Not all faces converted

Getting errors or found a bug when converting faces from a trained model? Post about them here


Forum rules

Read the FAQs and search the forum before posting a new topic.

This forum is for reporting errors with the Convert process. If you want to get tips, or better understand the Convert process, then you should look in the Convert Discussion forum.

Please mark any answers that fixed your problems so others can find the solutions.

Locked
User avatar
Artiros
Posts: 2
Joined: Fri Feb 19, 2021 8:23 pm

Not all faces converted

Post by Artiros »

Hi,

I have finally gotten my model trained to a level where I am happy and started converting. I noticed some frames where not converted and saw that the faces where missing (using the manual tool). Then I used some hours to add the missing masks hoping those frames would then be converted, however no such luck. I didn't spend much time fixing the landmarks, but made sure the masks (Unet-dfl) where OK.

Unfortunately I had no luck converting with the new masks in the alignment file. When using manual tool all frames I want to convert now have a mask (Unet-dfl), but for some reason they are just not converted. I'm sure there is something very obvious I'm missing here, and really appreciate if someone can push me in the right direction!

The reason so many faces are missing is likely that the video is blurry and it''s also an animated movie (Frozen). I'm quite impressed with the results as I know that blurry videos are not the best place to start, but I wanted to surprise my daughter transforming her to a Disney princess so had to try.. After 1 million iterations @batch size 16 (villain) it's pretty good.

Any ideas on how I can get the last frames converted? What am I missing after adding the masks?

User avatar
bryanlyon
Site Admin
Posts: 793
Joined: Fri Jul 12, 2019 12:49 am
Answers: 44
Location: San Francisco
Has thanked: 4 times
Been thanked: 218 times
Contact:

Re: Not all faces converted

Post by bryanlyon »

Usually what you're describing is due to "missing faces" in the fsa alignments file for the video. If you went through with the manual tool and added all the missing faces, then it should work correctly and convert all the faces. You may want to make sure that the faces you've marked actually include all the frames that you want swapped. Try looking specifically for ones where either the face you're looking for isn't found at all but also look for ones which are marked but show up "Upside down" or otherwise incorrectly in the preview image.

If you can find a specific frame that you know isn't getting converted, you can try attaching a screenshot of the manual tool of that frame to see if we notice anything wrong.

User avatar
Artiros
Posts: 2
Joined: Fri Feb 19, 2021 8:23 pm

Re: Not all faces converted

Post by Artiros »

I went into the manual tool again and noticed I had a few faces that were sideways, and multiple faces where the landmarks where way off. It seems I have underestimated the importance of the landmarks drasticly. To me it doesn't seem like the landmarks surrounding the face has too big of an impact, however the eyes/mouth/eyebrows are critical to get a good result.

So after seeing it frame by frame in the converted video I saw it did actually try to convert the images however it just didn't work very well because of the landmarks. I cleaned them up a bit and the problems are more or less gone. I will probably need to spend another 5 hours in the manual tool to improve the alignments a bit as some frames are very blurry due to improper landmarks (eyes), and also I get some double eyebrows which seems to be fixed with fixing the alignments also.

For my next project I'll make sure to spend more time in the manual tool before training, as I can imagine my training could have gone a lot faster if I didn't have some faces in there that had very poor landmarks/masks.

The tip on looking for faces that where sideways/upside down really helped identifying the issue, thanks a lot for your help!

Locked