Manual Alignment Bug?

Want to know about the Faceswap's Face Extraction process? Got tips, ideas or just want to learn about how it all works? Then this is the place for you


Forum rules

Read the FAQs and search the forum before posting a new topic.

This forum is for discussing tips and understanding the process involved for Extracting and preparing face sets for training a model in Faceswap.

If you have found a bug are having issues with the Extraction process not working, then you should post in the Extract Support forum.

Please mark any answers that fixed your problems so others can find the solutions.

Locked
User avatar
gravitation
Posts: 15
Joined: Fri May 07, 2021 6:14 am
Has thanked: 10 times
Been thanked: 1 time

Manual Alignment Bug?

Post by gravitation »

Ok. I am not sure if this was intended, a limitation of the program, or a bug. But I will try my best to explain.
I have example A that I simply want to extract and train so that I can replace example B with A. From what I understood, all I need to worry about for A is simply to make sure that the alignments are fixed using the Manual Tool.

I take a small video, extract it at 5 frames interval. Then Tools->Manual, I select the generated alignment file and select that video again and hit Manual. It loads, and the bottom thumbnails actually show everything accurately and correctly. Even if I enable the mesh overlay (F9), they are all aligned perfectly. However, it is when I click on any of them that the alignment severely jumps WAAAYY off. And when I iterate through all of them, they end up being very misaligned, but only if I view them in manual mode. If I don't touch them, they are fine. Now this happens for any video or any image set, on all the frames except the first one.

In one video, there are several scene changes of the environment. When I view them in the manual tool, they are all there and aligned perfectly. But when I click on them, not only do they misalign, but the image actually changes!

After some tinkering, I found out what the issue was. It appears that skipping frames doesn't play well with the manual tool. If I extract the entire video frame-by-frame, the manual tool shows correctly even if I iterate through the images. So here is the behavior that it has: If you extract a 400 frame video at 5 Frame interval yielding you 80 frames, and then launch Manual Tool, the manual tool will first display the correct frames. But when you press play, or click on any of them or just hold down X to iterate through them, they all get "misaligned", but in reality I don't think they do, I think that the manual tool will display the video frame-by-frame regardless. So if you view your extract frame 100, the manual tool will show you frame 20 of the video with the alignment set for 100!

Now I don't know if I am doing something wrong here, or missed something. I can't seem to find any answers, so I'm posting it here. Can someone help me figure this out? Or is this really something wrong? Just to be sure, I updated the app, and still gives me the same behavior.

User avatar
sp13
Posts: 15
Joined: Sat Apr 10, 2021 12:20 am
Has thanked: 3 times
Been thanked: 4 times

Re: Manual Alignment Bug?

Post by sp13 »

Yeah, when extracting with N > 1 like that, the first time I open the manual tool I let it load, then immediately close it without doing anything else. Then I open it again. It is always better the second time.

User avatar
torzdf
Posts: 2649
Joined: Fri Jul 12, 2019 12:53 am
Answers: 159
Has thanked: 128 times
Been thanked: 623 times

Re: Manual Alignment Bug?

Post by torzdf »

There are some slight bugs in the manual tool which I need to iron out. It's quite a complex bit of code though so a bit tough to track down.

The manual tool was also never designed with anything other than extracting every frame in mind though, which is why these kinds of things surface (it should be fine to use in that use case though)....

Thanks for the recreatable example, it will make this easier to bugfix

My word is final

User avatar
gravitation
Posts: 15
Joined: Fri May 07, 2021 6:14 am
Has thanked: 10 times
Been thanked: 1 time

Re: Manual Alignment Bug?

Post by gravitation »

sp13 wrote: Sun May 09, 2021 8:08 am

Yeah, when extracting with N > 1 like that, the first time I open the manual tool I let it load, then immediately close it without doing anything else. Then I open it again. It is always better the second time.

Hey it worked! I would like to confirm this for anyone else who might be having issues. The developer should pin this reply to to everyone having issues with this frame-by-frame manual tool issue until it has been resolved.

If you are using the Manual Tool on a picture set, or a video that has been extracted at more than 1 NFrame, then simply load the tool, and as soon as it loads successfully, immediately close it, and launch it again. You will see they align correctly with this method.

As for the developer, you are doing a great job man, no pressure from me (and hopefully anyone else) to get this done. Take your time and keep us posted.

User avatar
torzdf
Posts: 2649
Joined: Fri Jul 12, 2019 12:53 am
Answers: 159
Has thanked: 128 times
Been thanked: 623 times

Re: Manual Alignment Bug?

Post by torzdf »

This bug should now be fixed.

My word is final

Locked