Faceswap Forum

Hey everyone,

I have been learning a lot from this forum since last month. At first I hoped this would be a click and go, but lately I can see that it isnt. Thanks for all the work put in here, it has helped me learning all the basics of faceswaping.

I have been struggling to find information talking abot the loss functions. I found some papers (I dont understand nothing), some comentaries, but nothing decisive. This has driven me to write this post, in search of some information or links to posts.

I read that SSIM (Main), MAE(35), LPIPS(50) and FFL (100) are the most popular.

If im not wrong, loss function is how the NN knows if what it is doing is well or wrong, if the changes it is doing is going for better or worse, so it is obvious that is a key part for the process.

The things I would like to know are things like: "what changes you feel each function does to the NN?"(for example, when I turned on FFL, shines from sun would appear, instead of being mate. Or I read on the forum that MAE is some kind of random, so it would blur the image a little bit, things like that), "when do you apply each function?" (for example, since MAE blurs the image, is better for starting the training (I turn it down a little when I start recognizing the face (around 50-80k it maybe) and a bit more while the big training before I want to sharpen(100-400k it more or less). Or that LPIPS is good for definition, but it makes some artifacts(vgg16 should be the best, and dont make as much artifacts), so it is better for finishing the training).

Changing a little bit of subject, I also read that for the final touch is important to turn on No Warp, I just turn it on when I decide to start sharpening the swap. For sharpening or defining I also turn off MAE function and put on LPIPS (I have 8GB of VRAM on a 3070 so Im only capable of doing MS_SSIM, MAE, FFL, all that on Villain and a batch size of 6 at the same time, I change MAE for LPIPS for sharpening. I also want to learn about Phaze-A, but I thin Villain is more automatic). All this has came out of intuition from what I read and a little bit of experimentation.

I would apreciate any commentary about what you feel about function losses, advices about the final steps of training or better if you guys tell how is one of your processes of training, step by step please hehehe

Postdata: Im Spanish so excuse me for any orthographic errors hehehe

Discussions around loss functions are certainly tricky, because it takes so long to get results, and thoughts on what is happening can be entirely subjective.

If you haven't read this thread already, then I do recommend doing so:
viewtopic.php?t=2176

Thanks, thats what I was looking for.

Eventhough, I feel there has to be more. I want to lear how to fine tune the training, but thats the most difficult and abstract part of the training for sure. I will try making kind of a tree, making different changes for each brach. This will take so much time .
I will try posting what I learn.

Again, thanks.

Honestly, that would be very helpful. Side by side comparisons are the only real way to understand the influence of each function. However, unfortunately they take a long time to create.