Are Extracted faces downsized to match output Size?

If training is failing to start, and you are not receiving an error message telling you what to do, tell us about it here


Forum rules

Read the FAQs and search the forum before posting a new topic.

This forum is for reporting errors with the Training process. If you want to get tips, or better understand the Training process, then you should look in the Training Discussion forum.

Please mark any answers that fixed your problems so others can find the solutions.

Locked
User avatar
Scrapemist
Posts: 16
Joined: Sat Nov 05, 2022 2:26 pm
Has thanked: 8 times

Are Extracted faces downsized to match output Size?

Post by Scrapemist »

I ask because this is the case in DFL.

User avatar
torzdf
Posts: 2649
Joined: Fri Jul 12, 2019 12:53 am
Answers: 159
Has thanked: 128 times
Been thanked: 622 times

Re: Are Extracted faces downsized to match output Size?

Post by torzdf »

Not entirely sure what you mean. Internally, yes, all images are cropped, centered and sized to compare to model output. They have to be. The process (or any AI) wouldn't work otherwise.

Do we resize and save extracted images to model output size on disk? No. That would be entirely obsolete.

My word is final

User avatar
Scrapemist
Posts: 16
Joined: Sat Nov 05, 2022 2:26 pm
Has thanked: 8 times

Re: Are Extracted faces downsized to match output Size?

Post by Scrapemist »

I have an 4k video close-up, and extract at 1024px.
But I want to do experiments first on lower model res, say 256px.
So If its internally resized its not necessary to re-extract.

This means its better to extract face as close to the actual resolution of the source as possible if you want to reuse them in the future.
Is it then possible to mix extracted faces of different resolutions?

User avatar
torzdf
Posts: 2649
Joined: Fri Jul 12, 2019 12:53 am
Answers: 159
Has thanked: 128 times
Been thanked: 622 times

Re: Are Extracted faces downsized to match output Size?

Post by torzdf »

Ok. I think I follow more now.

Yes, you can extract at any size, however be aware that higher resolution extracts will lead to increased processing time in the data augmentation pipeline. Generally speaking this does not really matter too much, as normally higher resolution extracts are to train higher resolution models, so whilst data augmentation takes longer, so does putting a batch through the model, so it cancels each other out.

That said, the data augmentation pipeline is pretty highly optimized and has been tested with extracts in excess of 1024px with no impact on training speed. Testing is done on a pretty ancient 4 core Xeon (virtual machine), so you are unlikely to run into issues, but it's just something you may want to be aware of.

As long as all of the images in the training folder are faceswap extracted images, then the size of the image does not matter (can mix and match). Just be aware that you want to try to avoid having any of the training images being upscaled to meet the model output resolution.

You may also know, but just in case. You do not have to re-run the full extract process to generate training images at different sizes. You can use the alignments tool to extract training images straight from the alignments file.

My word is final

User avatar
Scrapemist
Posts: 16
Joined: Sat Nov 05, 2022 2:26 pm
Has thanked: 8 times

Re: Are Extracted faces downsized to match output Size?

Post by Scrapemist »

Interesting.

Besides output size, does the max input resolution of the encoder also have an effect on the resizing?
I see efficientnet_v2 had a max input of 480px.
So I assume the input to the encoder is resized to 480px if I feed it higher resolution faces.
Even if I have an output size of 512.

User avatar
torzdf
Posts: 2649
Joined: Fri Jul 12, 2019 12:53 am
Answers: 159
Has thanked: 128 times
Been thanked: 622 times

Re: Are Extracted faces downsized to match output Size?

Post by torzdf »

Yes, 2 images are created. One at model input size (for feeding into the model). One at model output size (for calculating loss).

My word is final

Locked