Alighment is slightly off on almost every frame

The Extraction process failing on you, and you aren't getting an error back with clear instructions? Tell us about it here


Forum rules

Read the FAQs and search the forum before posting a new topic.

This forum is for reporting errors with the Extraction process. If you want to get tips, or better understand the Extract process, then you should look in the Extract Discussion forum.

Please mark any answers that fixed your problems so others can find the solutions.

Locked
User avatar
unkempt
Posts: 81
Joined: Wed Dec 28, 2022 2:09 pm
Has thanked: 1 time
Been thanked: 9 times

Alighment is slightly off on almost every frame

Post by unkempt »

When I click through the manual tool, the face is detected fine and is precise shape-wise, but is shifted slightly in almost quite a lot of places that doesn't make sense. Face is pointing straight at the camera, clear details, etc. The detection is great really, just shifted slightly for some reason. I mean, I can manually click each frame and drag the face very slightly at which point it perfectly aligns without me doing anything (it can clearly detect it fine if I nudge it a little).

There's no way to select a series of frames and tell it to "recheck alignment" or something is there?

Last edited by unkempt on Sun Feb 19, 2023 2:24 pm, edited 2 times in total.
User avatar
torzdf
Posts: 2649
Joined: Fri Jul 12, 2019 12:53 am
Answers: 159
Has thanked: 128 times
Been thanked: 623 times

Re: Alighment is slightly off on almost every frame

Post by torzdf »

The explanation for what you are seeing is most likely not what you think it is...

I would guess the alignments are correct, but the way they are displaying in the manual tool is not correct. Your are seeing the alignment from the previous/next frame on the frame you are viewing.

Sometimes closing out of the manual tool and re-opening fixes this.

Short version of why this sometimes happens: Normally a slight issue with the video file, or maybe a small bug somewhere in the manual tool.

Ultimately, it is not possible to seek to an exact frame in a video file in a timely manner. However, this is exactly what we have to be able to do (the only exact way to do this would be to scan through every single frame, counting frames, on each frame change.... this is clearly not feasible).

The manual tool will seek to the correct frame based on the Presentation Timestamp (PTS) of the frame. The PTS is calculated the first time you launch the manual tool. However it has to make assumptions about the accuracy of the video file. Sadly, not all video files are created equally. I have put many safeguards in place to handle problematic videos, most of which you will have never noticed, because they are working. However, it is not possible for me to cover every possible issue.

My word is final

User avatar
unkempt
Posts: 81
Joined: Wed Dec 28, 2022 2:09 pm
Has thanked: 1 time
Been thanked: 9 times

Re: Alighment is slightly off on almost every frame

Post by unkempt »

Fair enough. I already reencoded this video once, but maybe I would get better results by doing so again with different settings. Either way, I'll see if I can tinker with it.

I doesn't seem to be a previous frame issue this time as the faces are correct other than placement and shift between frames. It's a bit irregular. What's odd is how easily it corrects the face positioning if I just click and drag the box in any direction - by one pixel even. It instantly corrects. I wonder what stopped it from doing that on the first run.

Last edited by unkempt on Sun Feb 19, 2023 4:59 pm, edited 1 time in total.
User avatar
MaxHunter
Posts: 193
Joined: Thu May 26, 2022 6:02 am
Has thanked: 177 times
Been thanked: 13 times

Re: Alighment is slightly off on almost every frame

Post by MaxHunter »

I'm just curious. Do you have the "Re-Align" option ticked off? If so, how many re-feeds are you giving it?

I've noticed that giving Re-Align more "passes" can actually cause this to happen for me. I use to give it the max passes (10) but lowered it because it was doing this to me. I don't know if this is just a placebo but it seems to be more accurate with lower passes, which kind of make sense because if the alignments are averaged out and you had one really bad alignment pass then it could throw off the final alignment.

Last edited by MaxHunter on Sun Feb 19, 2023 6:41 pm, edited 2 times in total.
User avatar
torzdf
Posts: 2649
Joined: Fri Jul 12, 2019 12:53 am
Answers: 159
Has thanked: 128 times
Been thanked: 623 times

Re: Alighment is slightly off on almost every frame

Post by torzdf »

That's placebo. There is no way upping re-feeds could cause that. Although, not really worth putting re-feeds above about 7.

I did have one video, a long time ago, which extracted fine (faces came out) but manual tool showed the faces well off (like it though the dimensions of the video were 4 times smaller than it was) so the faces were consistently offset, significantly. I never figured out what caused that though and moved on.

My word is final

User avatar
MaxHunter
Posts: 193
Joined: Thu May 26, 2022 6:02 am
Has thanked: 177 times
Been thanked: 13 times

Re: Alighment is slightly off on almost every frame

Post by MaxHunter »

Thanks. I've gone back and forth over that hypothesis.

I have it set at nine and still see moments of this problem but it seems less so than when set at ten.

I'm going to try your suggestion at seven. :)

User avatar
unkempt
Posts: 81
Joined: Wed Dec 28, 2022 2:09 pm
Has thanked: 1 time
Been thanked: 9 times

Re: Alighment is slightly off on almost every frame

Post by unkempt »

MaxHunter wrote: Sun Feb 19, 2023 6:38 pm

I'm just curious. Do you have the "Re-Align" option ticked off? If so, how many re-feeds are you giving it?

I've noticed that giving Re-Align more "passes" can actually cause this to happen for me. I use to give it the max passes (10) but lowered it because it was doing this to me. I don't know if this is just a placebo but it seems to be more accurate with lower passes, which kind of make sense because if the alignments are averaged out and you had one really bad alignment pass then it could throw off the final alignment.

Thanks for the response! I do have refeed and it's set to 6. I hadn't thought of lowering it... would less than 6 help do you think? Either way, thanks for the tip! I can try that with the next video that's a problem (but not this one because I already spent hours realigning 14k frames :D)

User avatar
unkempt
Posts: 81
Joined: Wed Dec 28, 2022 2:09 pm
Has thanked: 1 time
Been thanked: 9 times

Re: Alighment is slightly off on almost every frame

Post by unkempt »

Doh. I read the followup comment and I guess that's not the answer :)

User avatar
unkempt
Posts: 81
Joined: Wed Dec 28, 2022 2:09 pm
Has thanked: 1 time
Been thanked: 9 times

Re: Alighment is slightly off on almost every frame

Post by unkempt »

SOLVED IT

I realized the video had a bunch of scenes I didn't actually care about so I clipped out parts in Adobe Premiere and re-encoded it and it seems to have captured MOST faces (has some trouble with motion blur even if light) and the alignments are delightfully free of jitter.

Locked