Masking not working? Questions about masks (newbie)

Want to know about the Faceswap's Face Extraction process? Got tips, ideas or just want to learn about how it all works? Then this is the place for you


Forum rules

Read the FAQs and search the forum before posting a new topic.

This forum is for discussing tips and understanding the process involved for Extracting and preparing face sets for training a model in Faceswap.

If you have found a bug are having issues with the Extraction process not working, then you should post in the Extract Support forum.

Please mark any answers that fixed your problems so others can find the solutions.

Locked
User avatar
Absimiliard
Posts: 5
Joined: Sat Feb 06, 2021 8:31 am

Masking not working? Questions about masks (newbie)

Post by Absimiliard »

I just started dabbling with Faceswap this week. I have been playing with some different configurations, trying to get the swing of things. I hadn't generated any asks on the fist few projects and decided that they would help. I was able to generate a mask (Vgg-Obstructed) with the Mask Tool and was a little confused. In the output options I can choose between Combines, Masked and Mask. The Masked option says the output will be an "rgba image with the face masked". When I generated the mask png files with transparency were generated into the output folder specified.

I figured at this point that I should have generated these at the beginning so, on my next extraction, I selected Vgg-Obstructed as the Masker. During extraction I noticed that a there was a third pass for mask generation (as expected) but, upon completion, noticed that the output folder seemed to contain png files without transparency. I thought maybe the masked files were p[laced elsewhere but couldn't locate them so I assumed that maybe the alignment file was somehow containing the mask data. Upon training I noticed that areas that should have been masked were bleeding into the faces and guessed that the masks hadn't generated properly. I wasn't sure what went wrong so I saved the project and started a new one thinking that might resolve the issue. It did not as far as I can tell. Am I missing something?

User avatar
torzdf
Posts: 2649
Joined: Fri Jul 12, 2019 12:53 am
Answers: 159
Has thanked: 129 times
Been thanked: 622 times

Re: Masking not working? Questions about masks (newbie)

Post by torzdf »

No, you're doing nothing wrong.

The masks are not stored in the output images, but in the alignments file. This is because you can have multiple masks (and 2 are generated by default).

The easiest way to view them is with the manual tool. The "Output" option in the mask tool is just a mechanism for viewing the mask applied to a face. It serves no functional purpose within faceswap.

My personal preference is to not generate any extra masks during extraction, but generate them after I've cleaned up my alignments file. The reason for this is that when you change any landmarks (i.e. in the manual tool) the originally extracted masks are no longer valid, as the face image will have changed when you updated any alignments.

My word is final

User avatar
Absimiliard
Posts: 5
Joined: Sat Feb 06, 2021 8:31 am

Re: Masking not working? Questions about masks (newbie)

Post by Absimiliard »

Thanks very much for your reply. I've got that part figured now. I se that the mask tool can read or assign masks to an alignment file. I do have another mask question though...

How are the masks combined? By this I mean are they additive? If so then If I generate a mask with holes to exclude obstructions to the face isn't the effort removed when that mask is added to the component and extended mask during training or conversion as they will not have those exclusions?

Oops... nevermind I think I figured this out as well. It seems you can select the mask to be used in the Train settings. This would explain the issues I've been having

Locked