Very low EGs training

If training is failing to start, and you are not receiving an error message telling you what to do, tell us about it here


Forum rules

Read the FAQs and search the forum before posting a new topic.

This forum is for reporting errors with the Training process. If you want to get tips, or better understand the Training process, then you should look in the Training Discussion forum.

Please mark any answers that fixed your problems so others can find the solutions.

Locked
User avatar
Faildozer
Posts: 2
Joined: Sun Feb 16, 2020 1:08 pm

Very low EGs training

Post by Faildozer »

Hello, im currently running some training and the EGs numbers i get seem to be very low compared to others people's numbers i've seen.

Image

edit: forgot to add using the Dlight model

Code: Select all

============ System Information ============
encoding:            cp1252
git_branch:          master
git_commits:         7024047 Merge branch 'staging'. e6a2795 lib.gui.stats - Skip sessions with data corruption. cf7cf82 Merge branch 'staging'. 4cb2b40 bugfix: lib.alignments - mask summary datatype
gpu_cuda:            No global version found. Check Conda packages for Conda Cuda
gpu_cudnn:           No global version found. Check Conda packages for Conda cuDNN
gpu_devices:         GPU_0: GeForce RTX 2080
gpu_devices_active:  GPU_0
gpu_driver:          442.19
gpu_vram:            GPU_0: 8192MB
os_machine:          AMD64
os_platform:         Windows-10-10.0.18362-SP0
os_release:          10
py_command:          j:\faceswap/faceswap.py gui
py_conda_version:    conda 4.8.2
py_implementation:   CPython
py_version:          3.7.6
py_virtual_env:      True
sys_cores:           12
sys_processor:       Intel64 Family 6 Model 158 Stepping 10, GenuineIntel
sys_ram:             Total: 32684MB, Available: 20540MB, Used: 12144MB, Free: 20540MB

=============== Pip Packages ===============
absl-py==0.8.1
astor==0.8.0
certifi==2019.11.28
cloudpickle==1.3.0
cycler==0.10.0
cytoolz==0.10.1
dask==2.10.1
decorator==4.4.1
fastcluster==1.1.26
ffmpy==0.2.2
gast==0.2.2
google-pasta==0.1.8
grpcio==1.16.1
h5py==2.9.0
imageio==2.6.1
imageio-ffmpeg==0.3.0
joblib==0.14.1
Keras==2.2.4
Keras-Applications==1.0.8
Keras-Preprocessing==1.1.0
kiwisolver==1.1.0
Markdown==3.1.1
matplotlib==3.1.3
mkl-fft==1.0.15
mkl-random==1.1.0
mkl-service==2.3.0
networkx==2.4
numpy==1.17.4
nvidia-ml-py3==7.352.1
olefile==0.46
opencv-python==4.1.2.30
opt-einsum==3.1.0
pathlib==1.0.1
Pillow==6.0.0
protobuf==3.11.2
psutil==5.6.7
pyparsing==2.4.6
pyreadline==2.1
python-dateutil==2.8.1
pytz==2019.3
PyWavelets==1.1.1
pywin32==227
PyYAML==5.3
scikit-image==0.16.2
scikit-learn==0.22.1
scipy==1.4.1
six==1.14.0
tensorboard==2.0.0
tensorflow==1.15.0
tensorflow-estimator==1.15.1
termcolor==1.1.0
toolz==0.10.0
toposort==1.5
tornado==6.0.3
tqdm==4.42.1
Werkzeug==0.16.1
wincertstore==0.2
wrapt==1.11.2

============== Conda Packages ==============
# packages in environment at C:\Users\bever\MiniConda3\envs\faceswap:
#
# Name                    Version                   Build  Channel
_tflow_select             2.2.0                     eigen  
absl-py 0.8.1 py37_0
astor 0.8.0 py37_0
blas 1.0 mkl
ca-certificates 2020.1.1 0
certifi 2019.11.28 py37_0
cloudpickle 1.3.0 py_0
cycler 0.10.0 py37_0
cytoolz 0.10.1 py37he774522_0
dask-core 2.10.1 py_0
decorator 4.4.1 py_0
fastcluster 1.1.26 py37he350917_0 conda-forge ffmpeg 4.2 h6538335_0 conda-forge ffmpy 0.2.2 pypi_0 pypi freetype 2.9.1 ha9979f8_1
gast 0.2.2 py37_0
git 2.23.0 h6bb4b03_0
google-pasta 0.1.8 py_0
grpcio 1.16.1 py37h351948d_1
h5py 2.9.0 py37h5e291fa_0
hdf5 1.10.4 h7ebc959_0
icc_rt 2019.0.0 h0cc432a_1
icu 58.2 ha66f8fd_1
imageio 2.6.1 py37_0
imageio-ffmpeg 0.3.0 py_0 conda-forge intel-openmp 2020.0 166
joblib 0.14.1 py_0
jpeg 9b hb83a4c4_2
keras 2.2.4 0
keras-applications 1.0.8 py_0
keras-base 2.2.4 py37_0
keras-preprocessing 1.1.0 py_1
kiwisolver 1.1.0 py37ha925a31_0
libpng 1.6.37 h2a8f88b_0
libprotobuf 3.11.2 h7bd577a_0
libtiff 4.1.0 h56a325e_0
markdown 3.1.1 py37_0
matplotlib 3.1.1 py37hc8f65d3_0
matplotlib-base 3.1.3 py37h64f37c6_0
mkl 2020.0 166
mkl-service 2.3.0 py37hb782905_0
mkl_fft 1.0.15 py37h14836fe_0
mkl_random 1.1.0 py37h675688f_0
networkx 2.4 py_0
numpy 1.17.4 py37h4320e6b_0
numpy-base 1.17.4 py37hc3f5095_0
nvidia-ml-py3 7.352.1 pypi_0 pypi olefile 0.46 py37_0
opencv-python 4.1.2.30 pypi_0 pypi openssl 1.1.1d he774522_4
opt_einsum 3.1.0 py_0
pathlib 1.0.1 py37_1
pillow 6.2.1 py37hdc69c19_0
pip 20.0.2 py37_1
protobuf 3.11.2 py37h33f27b4_0
psutil 5.6.7 py37he774522_0
pyparsing 2.4.6 py_0
pyqt 5.9.2 py37h6538335_2
pyreadline 2.1 py37_1
python 3.7.6 h60c2a47_2
python-dateutil 2.8.1 py_0
pytz 2019.3 py_0
pywavelets 1.1.1 py37he774522_0
pywin32 227 py37he774522_1
pyyaml 5.3 py37he774522_0
qt 5.9.7 vc14h73c81de_0
scikit-image 0.16.2 py37h47e9c7a_0
scikit-learn 0.22.1 py37h6288b17_0
scipy 1.4.1 py37h9439919_0
setuptools 45.2.0 py37_0
sip 4.19.8 py37h6538335_0
six 1.14.0 py37_0
sqlite 3.31.1 he774522_0
tensorboard 2.0.0 pyhb38c66f_1
tensorflow 1.15.0 eigen_py37h9f89a44_0
tensorflow-base 1.15.0 eigen_py37h07d2309_0
tensorflow-estimator 1.15.1 pyh2649769_0
termcolor 1.1.0 py37_1
tk 8.6.8 hfa6e2cd_0
toolz 0.10.0 py_0
toposort 1.5 py_3 conda-forge tornado 6.0.3 py37he774522_3
tqdm 4.42.1 py_0
vc 14.1 h0510ff6_4
vs2015_runtime 14.16.27012 hf0eaf9b_1
werkzeug 0.16.1 py_0
wheel 0.34.2 py37_0
wincertstore 0.2 py37_0
wrapt 1.11.2 py37he774522_0
xz 5.2.4 h2fa13f4_4
yaml 0.1.7 hc54c509_2
zlib 1.2.11 h62dcd97_3
zstd 1.3.7 h508b16e_0 ================= Configs ================== --------- .faceswap --------- backend: nvidia --------- convert.ini --------- [color.color_transfer] clip: True preserve_paper: True [color.manual_balance] colorspace: HSV balance_1: 0.0 balance_2: 0.0 balance_3: 0.0 contrast: 0.0 brightness: 0.0 [color.match_hist] threshold: 99.0 [mask.box_blend] type: gaussian distance: 11.0 radius: 5.0 passes: 1 [mask.mask_blend] type: normalized kernel_size: 3 passes: 4 threshold: 4 erosion: 0.0 [scaling.sharpen] method: unsharp_mask amount: 150 radius: 0.3 threshold: 5.0 [writer.ffmpeg] container: mp4 codec: libx264 crf: 23 preset: medium tune: none profile: auto level: auto [writer.gif] fps: 25 loop: 0 palettesize: 256 subrectangles: False [writer.opencv] format: png draw_transparent: False jpg_quality: 75 png_compress_level: 3 [writer.pillow] format: png draw_transparent: False optimize: False gif_interlace: True jpg_quality: 75 png_compress_level: 3 tif_compression: tiff_deflate --------- extract.ini --------- [global] allow_growth: False [align.fan] batch-size: 64 [detect.cv2_dnn] confidence: 50 [detect.mtcnn] minsize: 20 threshold_1: 0.6 threshold_2: 0.7 threshold_3: 0.7 scalefactor: 0.709 batch-size: 8 [detect.s3fd] confidence: 70 batch-size: 4 [mask.unet_dfl] batch-size: 12 [mask.vgg_clear] batch-size: 12 [mask.vgg_obstructed] batch-size: 12 --------- gui.ini --------- [global] fullscreen: False tab: extract options_panel_width: 30 console_panel_height: 20 icon_size: 14 font: default font_size: 9 autosave_last_session: prompt timeout: 120 auto_load_model_stats: True --------- train.ini --------- [global] coverage: 68.75 mask_type: extended mask_blur_kernel: 3 mask_threshold: 4 learn_mask: False icnr_init: False conv_aware_init: False subpixel_upscaling: False reflect_padding: False penalized_mask_loss: True loss_function: mae learning_rate: 5e-05 [model.dfl_h128] lowmem: False [model.dfl_sae] input_size: 128 clipnorm: True architecture: df autoencoder_dims: 0 encoder_dims: 42 decoder_dims: 21 multiscale_decoder: False [model.dlight] features: fair details: good output_size: 128 [model.original] lowmem: False [model.realface] input_size: 64 output_size: 128 dense_nodes: 1536 complexity_encoder: 128 complexity_decoder: 512 [model.unbalanced] input_size: 128 lowmem: False clipnorm: True nodes: 1024 complexity_encoder: 128 complexity_decoder_a: 384 complexity_decoder_b: 512 [model.villain] lowmem: False [trainer.original] preview_images: 14 zoom_amount: 5 rotation_range: 10 shift_range: 5 flip_chance: 50 color_lightness: 30 color_ab: 8 color_clahe_chance: 50 color_clahe_max_size: 4

any ideas?

User avatar
deephomage
Posts: 33
Joined: Fri Jul 12, 2019 6:09 pm
Answers: 1
Has thanked: 2 times
Been thanked: 8 times

Re: Very low EGs training

Post by deephomage »

Your EGs are low because you're training with the CPU version of tensorflow rather than the GPU version, tensorflow-gpu. In your faceswap environment:

conda install tensorflow-gpu==1.15

Alternatively, delete the faceswap folder and run the installer again and choose Nvidia.

User avatar
Faildozer
Posts: 2
Joined: Sun Feb 16, 2020 1:08 pm

Re: Very low EGs training

Post by Faildozer »

I suspected that, i've installed it and made sure i picked nvidia 3 times, very strange, I'll try the command.

edit: YES! FINALLY!

this got me on the right path

Code: Select all

conda activate faceswap
//set the right env
conda install tensorflow-gpu==1.15
//installs tf gpu in the faceswap env

this is what i did and it it fixed it for me, maybe it will help someone else. tf gpu appeared to be installed in the conda folder but not in the env for some reason, my EGs has gone from 2 to 78, very happy.

Locked