Faceswap Forum

I have a couple of faces that are "highly asymetric". Specifically, they either contain facial features (noses, chins, eyes) or skin features (freckle bands, moles) that either don't look right or disappear completely i(n the case of moles and freckles) when trained using fliped images (NoFlip off) or neutral images (like wearing makeup). Also, many of the images are from selfies, so features in many photos are on wrong side even when flipping is off. Knowing what I know now I would have flipped the original photos before initial extraction, but...since I didn't... In each case, I only have about 500 or so "firm asymetric" images, and about 1500 total. I created two sets for initial and final training, and tried various amounts of training each. Most of the correctly oriented images are forward facing, which means they don't have enough variety to fix things like noses on upturned faces, eye position, etc. So only training using the Final set with NoFlip on gives good results regarding facial asymetry, but poor results regarding swap fidelity.

In order to try fixing this, I would like to add the 400 or so reversed selfie images to the Final set so I can train a greater variety using NoFlip. I am using a frames folder to hold the source images (versus MP4). So the questions:
1) Can I flip individual frames within the folder and tweak alignments without re-extracting everything from scratch? For example, can I use Update-Hashes after flipping? Or, probably more accurately, can I remove and purge the images to be flipped from the large Initial training set to create a separate set I can flip, re-hash and re-extract, and then merge them back into the original sets? Or am I better off re-extracting everything from scratch?
2) Are there any special tips you have for asymetric training?
Thanks!

Tekniklee wrote: ↑Wed Apr 22, 2020 4:54 am

1) Can I flip individual frames within the folder and tweak alignments without re-extracting everything from scratch? For example, can I use Update-Hashes after flipping? Or, probably more accurately, can I remove and purge the images to be flipped from the large Initial training set to create a separate set I can flip, re-hash and re-extract, and then merge them back into the original sets? Or am I better off re-extracting everything from scratch?

Assuming this is a training set only, I would (and this is off the top of my head, I apologize if I miss a step, or this doesn't work)

• Move the frames I need to flip out of the original frames folder into a new folder.

• Run the Remove Frames Alignments Tool job (pointing at the alignments file and the frames folder which still contains the frames you wish to keep) to remove the flipped frames from the alignments file.

• Flip the frames in the newly created folder containing the frames which you need to reprocess.

• Run extraction against this new folder to generate an alignments file for the faces you had to flip

• Run the Merge Alignments Tool job, pointing at the alignments file which contains original faces and alignments file which contains newly extracted flipped faces

• Put your flipped frames back into the original frames folder.

• Replace the alignments file in the original frames folder with the new merged file you have just created

• Run the Extract Alignments Tool job against the now merged folder, with the merged alignments file, to re-generate the full face set.

Tekniklee wrote: ↑Wed Apr 22, 2020 4:54 am

2) Are there any special tips you have for asymetric training?

Not something I have had experience with, but I would start with flipping enabled whilst the process is learning what a face is, then switch it off when it starts drilling down on the detail.

Thanks - that process makes sense. After thinking about it more I'll probably go in and try to collect more source images also. I've learned an awful lot about things like eyes since I culled the original images, so once I pull out all the existing frames I'll see if there are more to be had using a similar process. It's just, you know, to paraphrase Arthur Dent, facing down 10,000 images is daunting in the best of times, and aligning them doubly so....