Missing Faces 2 problems

Getting errors or found a bug when converting faces from a trained model? Post about them here


Forum rules

Read the FAQs and search the forum before posting a new topic.

This forum is for reporting errors with the Convert process. If you want to get tips, or better understand the Convert process, then you should look in the Convert Discussion forum.

Please mark any answers that fixed your problems so others can find the solutions.

Locked
User avatar
redson
Posts: 6
Joined: Wed Jun 22, 2022 2:56 am

Missing Faces 2 problems

Post by redson »

I ran the alignment Masker set to both VGG-Obstructed & Bisenet-FB

I did an conversion few parts of video mask goes away reverting back to source video.
I ran compare with missing faces to frames. I know every single frame has a face. But shows around 100 frames
not finding a face.

POSSIBLE BUG
When I was going to try to fix it manually. The Tools->Manual adjustment doesn't work had a video load issue because of 3D video alignment creates at least 2 faces per frame. When using manual says :mrgreen:
"There is a mismatch between the number of frames found in the video file (6143) and the number of frames found in the alignments file (12286)."

AUTOMATIC

I was wondering if there was a simpler way to cherry pick these missing frames specifically or even if I had to manually set these missing frames or is it only flipping one by one in manual mode?

Is there any way to load missing faces and flip one by one from those missing, compared to original alignment.
or could we type in the frame numbers

User avatar
torzdf
Posts: 2649
Joined: Fri Jul 12, 2019 12:53 am
Answers: 159
Has thanked: 128 times
Been thanked: 622 times

Re: Missing Faces 2 problems

Post by torzdf »

Ok, I'm not sure I'm fully understanding the problem.

The normal reason for missing masks is if you have done manual adjustments to the landmarks, in which case any NN generated masks will be deleted (they have to be, the aligned face that generated them is no longer valid). These missing masks can be regenerated with the mask tool.

"There is a mismatch between the number of frames found in the video file (6143) and the number of frames found in the alignments file (12286)."

This is odd to me, but it may be down to how the video file has stored 3D frames. I don't work much with 3D, but I would assume that a 3D frame would have the left and right image in a single frame. However, manual tool is reporting exactly double the number of frames in the alignments file.

This would suggest to me that the video file has somehow packed the left and right images into separate frames.

The manual tool is a tricky beast. It must be frame accurate, even being out by 1 frame will destroy any work you do. I have worked long and hard to get it as accurate as possible, and I would say it works 99% of the time. Unfortunately videos come in all sorts of codecs and formats, so it is not possible to cover 100% of eventualities.

When extracting, the video is simply iterated through from start to end, generating faces for the alignments file. This iteration generated 12,286 frames worth of data.

When using the manual tool, the video must be parsed end to end, to get the frame count. We do this using multiple techniques, including evaluating for missing keyframes, handling duped frames for 3:2 pull down, and trying to handle variable framerates, so we can get an accurate framecount of the video, plus the correct Presentation Timestamp to be able to jump accurately to any requested frame. If any one of these (and other) variables are incorrect, then the process will fail.

Ultimately, some variable in what your video is reporting is not playing nice with what the video actually generates when you iterate from start to end.

Re-encoding the video would be my first port of call. I don't know in what manner, as I do not work with 3D video.

My word is final

Locked