Scenario: Subject is the person who's face I want to be able to put on videos of other people.
- I gather source videos and photos of them that clearly show their face from various angles.
- Using the extraction guide, I let faceswap pull faces from the folder, sort out the bad ones, and am left with faces of various kinds
Question: I can't use the manual tool on photos so I have to use clear and simple photos without crazy angles and obstructions over the face, yes?
- Get target video (the face to be replaced).
- Extract target - make sure not to forget to skip frames so you don't end up with billions of them. Leave the mask OFF unless you think you won't need to do any manual adjustments (like when they have something over their face). -- edit. Nevermind. If you don't select your mask at this step, it might not be an option when doing manual later.
- Use the manual tool to look for faces that got weird at strong angles. Mask tool to deal with face obstructions(like if they're drinking from a bottle or put their hand in front of their face) - make sure to select your mask type right away or you'll do a lot of work for nothing)
- SAVE the changes!
- Delete the original faces you extracted and use the "Extract to folder" button next to the save to re-extract the same faces, but with the improved geometry and masks.
- Close the manual tool and open the MASK tool
- Use "only missing" to apply the same mask selected during the manual process to images you didn't fiddle with - target the folder of extracted faces
With this I now have my source folder of faces and my target folder of faces and can train.
QUESTION: would I want to mess with masking during the training phase? Training doesn't seem to care about custom masks at all
TRAINING
- Set inputs
- Run with "warp to landmarks" for a long time. Pretty much until you can clearly see eyes/teeth (or they're clearly not improving).
- Stop, deselect "warp to landmarks". Select "no warp" instead. Run again for a while.
QUESTION: Despite doing the manual process above, I'm not seeing the proper mask in the previews. It's as if my masks don't exist. Why? Aren't the masks part of the files? Or do they only come into play during conversion?
CONVERSION
- Repeat the process for extraction, except choose every frame instead of skipping
Question: if I use skip existing while extracting, it's telling me all fames are present, but when I open the manual tool, only about 100 are. What am I doing wrong?
- Use the manual tool to look for poor geometry and fix them
- Use the masking tool (this could take forever depending on the number of frames and how many have a face obstruction)
- Save. No need to extract this time because you're not using images; the alignments and masking are stored in the alignments file.
- Set conversion settings and roll. YOLO