Page 1 of 1
Extracting zoomed out frames? Faces too small per frame
Posted: Mon Jan 11, 2021 5:51 pm
by cosmico
Re: Extracting zoomed out frames? Faces too small per frame
Posted: Mon Jan 11, 2021 5:55 pm
by bryanlyon
This is expected operation. See https://github.com/deepfakes/faceswap/c ... 9868b0b662
You're running an old version while he has the current version is all.
Re: Extracting zoomed out frames? Faces too small per frame
Posted: Mon Jan 11, 2021 6:11 pm
by torzdf
More specifically:
https://github.com/deepfakes/faceswap/pull/1095
Don't know about the 2nd issue. Sounds like a glitch.
Re: Extracting zoomed out frames? Faces too small per frame
Posted: Tue Jan 12, 2021 4:37 pm
by cosmico
I'm don't think either of you understand the issue. Perhaps I didn't write it clearly.
The issue is that his newer version is the one messed up one and giving faces that are way too zoomed out, while my older version is giving the correct appropriate sized frames. And as far as I can tell, every single one of our settings related to extraction are the same.
Re: Extracting zoomed out frames? Faces too small per frame
Posted: Tue Jan 12, 2021 4:38 pm
by bryanlyon
If they look like the samples given in those links, then your friend's extract is correct. If they're more zoomed out than that, please post some samples so we can see what you mean.
Re: Extracting zoomed out frames? Faces too small per frame
Posted: Thu Jan 14, 2021 6:20 pm
by cosmico
Hey guys, torzdf and bryanlyon. Thanks for both being so great and willing to help everyone out. You two really what makes faceswap work.
I just want absolute, crystal clear, 1000%, beyond any doubt, understanding on what your saying.
.
.
So you are saying that his update to date program that is extracting 256 sized image,with 77% coverage, and results in an frames like these two below were the faces are really zoomed out, is correct?
.
.
And that my slightly older version, when extracting the same exact video with the same settings, and gives a frame that the face takes up the majority of the frame is out of date? like the one below?
I think the reason I'm kind of confused and asking for an ridiculous amount of confirmation is for several reasons. I've ran faceswap for close to a 1.25 years now, lots of updates and new versions, and faceswap always extracted frames like mine currently is rather than his zoomed out one, so this seemed more like an error than a new change. Another reason why it seems like an error is ....the face being the majority of the frame seems like a good thing.....does it not? If I'm trying to make a swap on a trainer where the input/output are both say 172 pixels, and I'm using 256px extracted frames, ...well that's totally fine because the actual face takes up like 176pixels in that 256px frame, where as in his, it feels like his face is only 128 pixels big in 256 sized framed....which seems like it would be worse and thus this seems like an error. Any chance you could quell my worries and give me more understanding? Sorry for being so high maintenance about this issue.
Re: Extracting zoomed out frames? Faces too small per frame
Posted: Thu Jan 14, 2021 6:25 pm
by bryanlyon
Yes, this is deliberate. See the link that Torzdf posted as to why.
Re: Extracting zoomed out frames? Faces too small per frame
Posted: Thu Jan 14, 2021 6:26 pm
by torzdf
The zoomed out version is correct.
Our default extract size is now 512px (This means that the same facial area from your version within the new extract is 308px, so the sizes are actually comparable)
Why did we do this? As you can see from your crop, the forehead is nearly always cropped out, and sometimes so is the chin.
The new extract brings out the entire head, which gives us scope in the future for different masking and head based swaps (which may or may not be possible, but either way needs a bigger extract).
We do not want multiple extract types, because it's confusing.
When you train a model, you can still train with "legacy" centering. This is basically exactly the same crop as you see in your zoomed in version.
If you look at the PR I linked to above, the boxes show what is fed into the model for "legacy" or "face" centering. At this moment in time the full extracted image is never fed into the model, rather a sub-crop from the extract is fed in.
Re: Extracting zoomed out frames? Faces too small per frame
Posted: Thu Jan 14, 2021 10:30 pm
by cosmico
Thank you, this explains everything perfectly, and I even understand you reasoning for doing it. Unfortunately I'm only able to mark one of your comments as the answer, but thank you to both of you.