Faceswap Forum

I'm at 975,000it's and I can't get the loss value below .071. In over 150+K it's been working in the .07s. It can't have reached convergence yet, can it? It just seems like it should be down in the threes, or the fours before reaching that point. Is this when I freeze the weights? Use No Warp?

And, while we're at it...

The previews are still grey/blueish. Is that normal?

Open to opinions?

Using a Phase A Stojo/Icarus Mix 256 Model

MaxHunter wrote: ↑Tue Sep 06, 2022 5:28 am

I'm at 975,000it's and I can't get the loss value below .071. In over 150+K it's been working in the .07s. It can't have reached convergence yet, can it? It just seems like it should be down in the threes, or the fours before reaching that point. Is this when I freeze the weights? Use No Warp?

Loss values are meaningless. All that matters is that they are going down. Don't look at the raw numbers

The previews are still grey/blueish. Is that normal?

Not sure what you mean by this. Are they all a solid color? If so, you model is not working.

This is what I'm talking about. The image in the left is the training reconstruction, the image on the right is the swap preview. All of my preview images are like this. I just figured it had something to do with one of the loss functions. It doesn't seem to affect the actual swap though.

Also, since I have your attention. Has anyone considered having a "convergence alert"? Maybe an automatic alert that warns the users that loss hasn't dropped in "x" amount of iterations? I mean, if you had something like that think of all the problems and questions about loss values that would cease from the noobs like me.

I think this is an RGB/BGR color order issue with the previews. I will investigate

This should be fixed in latest commit. Please update

I updated and it seems to still be there. Maybe the changes will take effect on a new model? I'll assume I shouldn't worry too much about it because it hasn't been an issue so far when swapping. I think a question I do have is, will this affect the color of the model?

No. It is just the preview window, it's not model connected. Bit odd that an update didn't fix it, I will have to double check.

I started training w/DNY512 and encoder fs original, rather than v2_l and the Stojo/Icarus Model. But I also turned off the learn Mask option. I'm beginning to wonder if the learn Mask is causing a lot of the problems, and one of the problems I think it's causing is this discoloration. Everything else is the same, loss functions, etc.

I haven't had any problems with ooms, nans, or this discoloration that I was having using the Stojo setting. Is it possible the Learn Mask could be causing problems?

It shouldn't make a difference.

The reason for the color weirdness is that most models are in BGR color order (openCVs color format) but some are in RGB order (most keras encoders), so we have to display the previews correctly (as I said, the model is fine, this just impact previews). Basically, in those examples you give above, the red and blue channels have been switched on the background and in the NN generated face. The image which shows the original face is in the correct channel order.

Oh my gosh, thanks for explaining that fully. I know it's hard and tedious to answer all these questions, but when you explain what's happening "under the hood," it makes it so much easier to understand other questions not asked.

Hate to add another post on this, as we dealt with this at an earlier time but I can't find the old post.

I started a new model based on STOJO, at 512, using efficienetv2 -L (a 23Gb model .) It seems the color scale hasn't been fixed. I brought this up months ago but moved on to the DNY512 model. Now that I'm back I was wondering if anyone can fix it because it makes it really hard to figure out when it's done.

MaxHunter wrote: ↑Wed Feb 01, 2023 11:33 pm

Hate to add another post on this, as we dealt with this at an earlier time but I can't find the old post.

I started a new model based on STOJO, at 512, using efficienetv2 -L (a 23Gb model .) It seems the color scale hasn't been fixed. I brought this up months ago but moved on to the DNY512 model. Now that I'm back I was wondering if anyone can fix it because it makes it really hard to figure out when it's done.

This issue was fixed, quite some time ago.

Are you up to date? If so please provide new screengrabs

Yep. Up to date.
Here you go...
(Also, in case anyone cares, this is STOJO - with a few modifications - at 512, 100K in. I'm shocked at how good it looks already!)

I don't know if this helps, but I just noticed there's a difference between these pics and previous pics. You'll notice only the face is off this time, where as before the entire photo was off. Not sure if that helps. Also, here are my parameters (And if you see anything you'd change please let me know.):

================= Configs ==================

--------- train.ini ---------

[global]
centering:                face
coverage:                 87.5
icnr_init:                True
conv_aware_init:          True
optimizer:                adam
learning_rate:            1e-05
epsilon_exponent:         -7
autoclip:                 True
reflect_padding:          False
allow_growth:             False
mixed_precision:          True
nan_protection:           True
convert_batchsize:        20

[global.loss]
loss_function:            ms_ssim
loss_function_2:          logcosh
loss_weight_2:            50
loss_function_3:          lpips_vgg16
loss_weight_3:            35
loss_function_4:          ffl
loss_weight_4:            100
mask_loss_function:       mse
eye_multiplier:           3
mouth_multiplier:         4
penalized_mask_loss:      True
mask_type:                bisenet-fp_face
mask_blur_kernel:         3
mask_threshold:           4
learn_mask:               True

[model.phaze_a]
output_size:              512
shared_fc:                none
enable_gblock:            True
split_fc:                 True
split_gblock:             False
split_decoders:           True
enc_architecture:         efficientnet_v2_l
enc_scaling:              100
enc_load_weights:         True
bottleneck_type:          dense
bottleneck_norm:          none
bottleneck_size:          512
bottleneck_in_encoder:    True
fc_depth:                 1
fc_min_filters:           1280
fc_max_filters:           1280
fc_dimensions:            8
fc_filter_slope:          -0.5
fc_dropout:               0.0
fc_upsampler:             upscale_hybrid
fc_upsamples:             1
fc_upsample_filters:      1280
fc_gblock_depth:          3
fc_gblock_min_nodes:      512
fc_gblock_max_nodes:      512
fc_gblock_filter_slope:   -0.5
fc_gblock_dropout:        0.0
dec_upscale_method:       upscale_hybrid
dec_upscales_in_fc:       0
dec_norm:                 none
dec_min_filters:          160
dec_max_filters:          640
dec_slope_mode:           full
dec_filter_slope:         -0.33
dec_res_blocks:           1
dec_output_kernel:        3
dec_gaussian:             True
dec_skip_last_residual:   False
freeze_layers:            keras_encoder
load_layers:              encoder
fs_original_depth:        4
fs_original_min_filters:  128
fs_original_max_filters:  1024
fs_original_use_alt:      False
mobilenet_width:          1.0
mobilenet_depth:          1
mobilenet_dropout:        0.001
mobilenet_minimalistic:   False

[trainer.original]
preview_images:           14
zoom_amount:              5
rotation_range:           10
shift_range:              5
flip_chance:              50
color_lightness:          30
color_ab:                 8
color_clahe_chance:       50
color_clahe_max_size:     4

Added bug tag, Will look over the weekend

Just in case this was supposed to be fixed in the past couple updates. Colors are still messed up.

Not checked yet. Will update this thread when I do.

Fixed.