manual tool - purpose of third window?

Want to know about the Faceswap's Face Extraction process? Got tips, ideas or just want to learn about how it all works? Then this is the place for you


Forum rules

Read the FAQs and search the forum before posting a new topic.

This forum is for discussing tips and understanding the process involved for Extracting and preparing face sets for training a model in Faceswap.

If you have found a bug are having issues with the Extraction process not working, then you should post in the Extract Support forum.

Please mark any answers that fixed your problems so others can find the solutions.

Locked
User avatar
korrupt78
Posts: 50
Joined: Wed Jan 29, 2020 1:34 am
Has thanked: 2 times
Been thanked: 1 time

manual tool - purpose of third window?

Post by korrupt78 »

Finally trying out the GUI in order to learn how to improve alignments for conversion, based on the guide in this subforum. I figured out how to navigate, adjust bounding boxes, and save, which is great - but I'm confused about the purpose of the third window.

What I mean is, when I run the manual tool from main GUI window, it pops up three new windows:
1) tall rectangle with keyboard shortcuts
2) large window with full size frame of video being worked on
3) wide rectangle with four adjacent boxes, and a thumbnail of the current frame in the left-most box (the other three are black)

I don't understand the purpose of that third window.

User avatar
korrupt78
Posts: 50
Joined: Wed Jan 29, 2020 1:34 am
Has thanked: 2 times
Been thanked: 1 time

Re: manual tool - purpose of third window?

Post by korrupt78 »

Still working on my first video (900 frames), realigned most of them manually already, but there are about a dozen that for some reason I can't make work.

Where ever on the face I click, the bounding box ends up being much smaller than I need. In 98% of cases I click around the mouth/nose area and it works perfectly, but for these dozen frames - which aren't even that different from the other ones - the box is much tinier, and I can't figure out how to expand it.

I see in the Help section that "middle click -> bounding box resize", but I can't figure out how that works. It just seems to produce nonsense when I try it...

User avatar
korrupt78
Posts: 50
Joined: Wed Jan 29, 2020 1:34 am
Has thanked: 2 times
Been thanked: 1 time

Re: manual tool - purpose of third window?

Post by korrupt78 »

Arg. Problem #3:

I just finished manually adjusting the alignment (or deleting the frame) across 900 frames (30s of video) and saved them. I then ran Extract to see the results as a set of images, and there's where I found surprises.

For example, when I'm in the Manual tool, if I'm looking at frame #213, I see the bottom half of a face correctly outlined by the boxes and points. (the top half isn't in this frame)

However, after Extract, the image file for frame #213 is just a forehead with some hair. Its almost as if its a completely different frame...

User avatar
korrupt78
Posts: 50
Joined: Wed Jan 29, 2020 1:34 am
Has thanked: 2 times
Been thanked: 1 time

Re: manual tool - purpose of third window?

Post by korrupt78 »

Arg. Problem #4.

In the meantime, decided to try converting the video using the work I'd already completed in the current version of my alignments file, to see how close I am.

Total fail. Almost every frame errored with something like:

02/12/2020 12:59:35 ERROR Failed to convert image: 'test_000130.png'. Reason: 'extended'

I'm pretty lost at this point.

User avatar
torzdf
Posts: 2649
Joined: Fri Jul 12, 2019 12:53 am
Answers: 159
Has thanked: 128 times
Been thanked: 623 times

Re: manual tool - purpose of third window?

Post by torzdf »

Manual tool is being reworked. However:

korrupt78 wrote: Wed Feb 12, 2020 7:22 pm

3) wide rectangle with four adjacent boxes, and a thumbnail of the current frame in the left-most box (the other three are black)

I don't understand the purpose of that third window.

It's a zoomed in box of the faces in the frame. If there are more than one face in the frame, the other boxes will fill. Just makes it easier to see on smaller faces

I see in the Help section that "middle click -> bounding box resize", but I can't figure out how that works. It just seems to produce nonsense when I try it...

Hold down the middle mouse button in the box and drag up/down to adjust the box height and left and right to adjust the box width.

However, after Extract, the image file for frame #213 is just a forehead with some hair. Its almost as if its a completely different frame...

Would need to see an example. Does this happen on every frame?

02/12/2020 12:59:35 ERROR Failed to convert image: 'test_000130.png'. Reason: 'extended'

That looks like a bug. In the meantime, you can use the mask tool to re-generate the extended mask for your alignments file and it should fix it.

My word is final

User avatar
korrupt78
Posts: 50
Joined: Wed Jan 29, 2020 1:34 am
Has thanked: 2 times
Been thanked: 1 time

Re: manual tool - purpose of third window?

Post by korrupt78 »

torzdf wrote: Wed Feb 12, 2020 10:01 pm

Manual tool is being reworked.

Nice. I love progress.

torzdf wrote: Wed Feb 12, 2020 10:01 pm

Hold down the middle mouse button in the box and drag up/down to adjust the box height and left and right to adjust the box width.

Ok, with that understanding a little patience, I got that to work. I find it unintuitive - especially when the results lag a second or two behind the movement - but I see now how to carefully use it to improve landmark detection when the initial bounding box isn't appropriately sized or shaped. (left/down -> decrease dimension, right/up -> increase dimension)

It's especially weird when you drag left (to shrink width) which results in the left side of the box moving right. Feels wrong. :) Maybe at some point I'll dive into the code and change those bindings to arrow keys.

(Still working through the other issues.)

User avatar
torzdf
Posts: 2649
Joined: Fri Jul 12, 2019 12:53 am
Answers: 159
Has thanked: 128 times
Been thanked: 623 times

Re: manual tool - purpose of third window?

Post by torzdf »

I wouldn't bother, It's being replaced with something much better,

Yeah, cv2 is a bit fiddly for that kind of thing, also was forced to use middle mouse button because of stupid cv2 default bindings.

My word is final

User avatar
korrupt78
Posts: 50
Joined: Wed Jan 29, 2020 1:34 am
Has thanked: 2 times
Been thanked: 1 time

Re: manual tool - purpose of third window?

Post by korrupt78 »

torzdf wrote: Thu Feb 13, 2020 10:26 pm

I wouldn't bother, It's being replaced with something much better,

Any guess when that'll be released? If it's really soon (days or weeks), it would make sense for me to avoid investing any more time practicing with and building on the existing tool and spend my time on other factors until the new tool arrives.

User avatar
torzdf
Posts: 2649
Joined: Fri Jul 12, 2019 12:53 am
Answers: 159
Has thanked: 128 times
Been thanked: 623 times

Re: manual tool - purpose of third window?

Post by torzdf »

More likely weeks, but it will be released to Patreons first.

My word is final

User avatar
korrupt78
Posts: 50
Joined: Wed Jan 29, 2020 1:34 am
Has thanked: 2 times
Been thanked: 1 time

Re: manual tool - purpose of third window?

Post by korrupt78 »

torzdf wrote: Thu Feb 13, 2020 11:42 pm

More likely weeks, but it will be released to Patreons first.

Cool - just joined.

Locked