Attempting to restart training session using dlight processes one frame then quits
Posted: Mon Nov 04, 2024 8:11 pm
I am trying to restart a model I have been training on for the past few days. It loads everything up, processes a single frame then exits, without as much as a whimper or an error message. As per the deepfake admin requirements, here are the specs from my system, collected from deepfake help:
Code: Select all
============ System Information ============
backend: nvidia
encoding: cp1252
git_branch: Not Found
git_commits: Not Found
gpu_cuda: No global version found. Check Conda packages for Conda Cuda
gpu_cudnn: No global version found. Check Conda packages for Conda cuDNN
gpu_devices: GPU_0: NVIDIA GeForce GTX 760
gpu_devices_active: GPU_0
gpu_driver: 475.14
gpu_vram: GPU_0: 2048MB (202MB free)
os_machine: AMD64
os_platform: Windows-10-10.0.19045-SP0
os_release: 10
py_command: C:\Users\Test User\faceswap/faceswap.py gui
py_conda_version: conda 24.9.2
py_implementation: CPython
py_version: 3.10.15
py_virtual_env: True
sys_cores: 20
sys_processor: Intel64 Family 6 Model 62 Stepping 4, GenuineIntel
sys_ram: Total: 32677MB, Available: 15483MB, Used: 17194MB, Free: 15483MB
=============== Pip Packages ===============
============== Conda Packages ==============
# packages in environment at C:\Users\Test User\MiniConda3\envs\faceswap:
#
# Name Version Build Channel
absl-py 2.1.0 pypi_0 pypi
aom 3.7.1 h63175ca_0 conda-forge
astunparse 1.6.3 pypi_0 pypi
blas 1.0 mkl
brotli 1.0.9 h2bbff1b_8
brotli-bin 1.0.9 h2bbff1b_8
bzip2 1.0.8 h2bbff1b_6
ca-certificates 2024.9.24 haa95532_0
cachetools 5.5.0 pypi_0 pypi
certifi 2024.8.30 pypi_0 pypi
charset-normalizer 3.4.0 pypi_0 pypi
colorama 0.4.6 py310haa95532_0
contourpy 1.2.0 py310h59b6b97_0
cudatoolkit 11.2.2 h7d7167e_13 conda-forge
cudnn 8.1.0.77 h3e0f4f4_0 conda-forge
cycler 0.11.0 pyhd3eb1b0_0
dav1d 1.2.1 hcfcfb64_0 conda-forge
expat 2.6.3 he0c23c2_0 conda-forge
fastcluster 1.2.6 py310hbbd805c_4 conda-forge
ffmpeg 6.1.0 gpl_h0859920_103 conda-forge
ffmpy 0.3.0 pyhb6f538c_0 conda-forge
flatbuffers 24.3.25 pypi_0 pypi
font-ttf-dejavu-sans-mono 2.37 hab24e00_0 conda-forge
font-ttf-inconsolata 3.000 h77eed37_0 conda-forge
font-ttf-source-code-pro 2.038 h77eed37_0 conda-forge
font-ttf-ubuntu 0.83 h77eed37_3 conda-forge
fontconfig 2.14.2 hbde0cde_0 conda-forge
fonts-conda-ecosystem 1 0 conda-forge
fonts-conda-forge 1 0 conda-forge
fonttools 4.51.0 py310h2bbff1b_0
freetype 2.12.1 ha860e81_0
gast 0.4.0 pypi_0 pypi
giflib 5.2.2 h7edc060_0
git 2.45.2 haa95532_1
google-auth 2.35.0 pypi_0 pypi
google-auth-oauthlib 0.4.6 pypi_0 pypi
google-pasta 0.2.0 pypi_0 pypi
grpcio 1.67.0 pypi_0 pypi
h5py 3.12.1 pypi_0 pypi
icc_rt 2022.1.0 h6049295_2
icu 73.1 h6c2663c_0
idna 3.10 pypi_0 pypi
imageio 2.33.1 py310haa95532_0
imageio-ffmpeg 0.5.1 pypi_0 pypi
intel-openmp 2023.1.0 h59b6b97_46320
joblib 1.4.2 py310haa95532_0
jpeg 9e h827c3e9_3
keras 2.10.0 pypi_0 pypi
keras-preprocessing 1.1.2 pypi_0 pypi
kiwisolver 1.4.4 py310hd77b12b_0
krb5 1.20.1 h5b6d351_0
lerc 3.0 hd77b12b_0
libbrotlicommon 1.0.9 h2bbff1b_8
libbrotlidec 1.0.9 h2bbff1b_8
libbrotlienc 1.0.9 h2bbff1b_8
libclang 18.1.1 pypi_0 pypi
libclang13 14.0.6 default_h8e68704_1
libdeflate 1.17 h2bbff1b_1
libexpat 2.6.3 he0c23c2_0 conda-forge
libffi 3.4.4 hd77b12b_1
libiconv 1.17 hcfcfb64_2 conda-forge
libopus 1.3.1 h8ffe710_1 conda-forge
libpng 1.6.39 h8cc25b3_0
libpq 12.17 h906ac69_0
libsqlite 3.46.1 h2466b09_0 conda-forge
libtiff 4.5.1 hd77b12b_0
libwebp 1.3.2 hbc33d0d_0
libwebp-base 1.3.2 h3d04722_1
libxml2 2.12.7 h283a6d9_1 conda-forge
libzlib 1.2.13 h2466b09_6 conda-forge
libzlib-wapi 1.2.13 h2466b09_6 conda-forge
lz4-c 1.9.4 h2bbff1b_1
markdown 3.7 pypi_0 pypi
markupsafe 3.0.2 pypi_0 pypi
matplotlib 3.9.2 py310haa95532_0
matplotlib-base 3.9.2 py310he19b0ae_0
mkl 2023.1.0 h6b88ed4_46358
mkl-service 2.4.0 py310h2bbff1b_1
mkl_fft 1.3.10 py310h827c3e9_0
mkl_random 1.2.7 py310hc64d2fc_0
numexpr 2.8.7 py310h2cd9be0_0
numpy 1.26.4 py310h055cbcc_0
numpy-base 1.26.4 py310h65a83cf_0
nvidia-ml-py 12.560.30 pyhd8ed1ab_0 conda-forge
oauthlib 3.2.2 pypi_0 pypi
opencv-python 4.10.0.84 pypi_0 pypi
openh264 2.4.0 h63175ca_0 conda-forge
openssl 3.3.2 h2466b09_0 conda-forge
opt-einsum 3.4.0 pypi_0 pypi
packaging 24.1 py310haa95532_0
pillow 9.4.0 py310hd77b12b_1
pip 24.2 py310haa95532_0
ply 3.11 py310haa95532_0
protobuf 3.19.6 pypi_0 pypi
psutil 5.9.0 py310h2bbff1b_0
pyasn1 0.6.1 pypi_0 pypi
pyasn1-modules 0.4.1 pypi_0 pypi
pybind11-abi 5 hd3eb1b0_0
pyparsing 3.1.2 py310haa95532_0
pyqt 5.15.10 py310hd77b12b_0
pyqt5-sip 12.13.0 py310h2bbff1b_0
python 3.10.15 h4607a30_1
python-dateutil 2.9.0post0 py310haa95532_2
python_abi 3.10 2_cp310 conda-forge
pywin32 305 py310h2bbff1b_0
pywinpty 2.0.2 py310h5da7b33_0
qt-main 5.15.2 h19c9488_10
requests 2.32.3 pypi_0 pypi
requests-oauthlib 2.0.0 pypi_0 pypi
rsa 4.9 pypi_0 pypi
scikit-learn 1.5.1 py310hc64d2fc_0
scipy 1.13.1 py310h8640f81_0
setuptools 75.1.0 py310haa95532_0
sip 6.7.12 py310hd77b12b_0
six 1.16.0 pyhd3eb1b0_1
sqlite 3.45.3 h2bbff1b_0
svt-av1 1.7.0 h63175ca_0 conda-forge
tbb 2021.8.0 h59b6b97_0
tensorboard 2.10.1 pypi_0 pypi
tensorboard-data-server 0.6.1 pypi_0 pypi
tensorboard-plugin-wit 1.8.1 pypi_0 pypi
tensorflow 2.10.1 pypi_0 pypi
tensorflow-estimator 2.10.0 pypi_0 pypi
tensorflow-io-gcs-filesystem 0.31.0 pypi_0 pypi
termcolor 2.5.0 pypi_0 pypi
threadpoolctl 3.5.0 py310h9909e9c_0
tk 8.6.14 h0416ee5_0
tomli 2.0.1 py310haa95532_0
tornado 6.4.1 py310h827c3e9_0
tqdm 4.66.5 py310h9909e9c_0
typing-extensions 4.12.2 pypi_0 pypi
tzdata 2024b h04d1e81_0
ucrt 10.0.22621.0 h57928b3_1 conda-forge
unicodedata2 15.1.0 py310h2bbff1b_0
urllib3 2.2.3 pypi_0 pypi
vc 14.40 h2eaa2aa_1
vc14_runtime 14.40.33810 hcc2c482_22 conda-forge
vs2015_runtime 14.40.33810 h3bf8584_22 conda-forge
werkzeug 3.0.4 pypi_0 pypi
wheel 0.44.0 py310haa95532_0
winpty 0.4.3 4
wrapt 1.16.0 pypi_0 pypi
x264 1!164.3095 h8ffe710_2 conda-forge
x265 3.5 h2d74725_3 conda-forge
xz 5.4.6 h8cc25b3_1
zlib 1.2.13 h2466b09_6 conda-forge
zlib-wapi 1.2.13 h2466b09_6 conda-forge
zstd 1.5.6 h8880b57_0
================= Configs ==================
--------- .faceswap ---------
backend: nvidia
--------- convert.ini ---------
[color.color_transfer]
clip: True
preserve_paper: True
[color.manual_balance]
colorspace: HSV
balance_1: 0.0
balance_2: 0.0
balance_3: 0.0
contrast: 0.0
brightness: 0.0
[color.match_hist]
threshold: 90.0
[mask.mask_blend]
type: normalized
kernel_size: 3
passes: 4
threshold: 4
erosion: 100.0
erosion_top: 100.0
erosion_bottom: 34.7
erosion_left: 0.0
erosion_right: 0.0
[scaling.sharpen]
method: gaussian
amount: 229
radius: 0.8
threshold: 5.0
[writer.ffmpeg]
container: mp4
codec: libx264
crf: 23
preset: medium
tune: None
profile: auto
level: auto
skip_mux: False
[writer.gif]
fps: 25
loop: 0
palettesize: 256
subrectangles: False
[writer.opencv]
format: png
draw_transparent: False
separate_mask: False
jpg_quality: 75
png_compress_level: 3
[writer.patch]
start_index: 0
index_offset: 0
number_padding: 6
include_filename: True
face_index_location: before
origin: bottom-left
empty_frames: blank
json_output: False
separate_mask: False
bit_depth: 16
format: png
png_compress_level: 3
tiff_compression_method: lzw
[writer.pillow]
format: png
draw_transparent: False
separate_mask: False
optimize: False
gif_interlace: True
jpg_quality: 75
png_compress_level: 3
tif_compression: tiff_deflate
--------- extract.ini ---------
[global]
allow_growth: False
aligner_min_scale: 0.07
aligner_max_scale: 2.0
aligner_distance: 22.5
aligner_roll: 45.0
aligner_features: True
filter_refeed: True
save_filtered: False
realign_refeeds: True
filter_realign: True
[align.external]
file_name: import.json
origin: top-left
4_point_centering: head
[align.fan]
batch-size: 12
[detect.cv2_dnn]
confidence: 50
[detect.external]
file_name: import.json
origin: top-left
[detect.mtcnn]
minsize: 20
scalefactor: 0.709
batch-size: 8
cpu: True
threshold_1: 0.6
threshold_2: 0.7
threshold_3: 0.7
[detect.s3fd]
confidence: 70
batch-size: 4
[mask.bisenet_fp]
batch-size: 8
cpu: False
weights: faceswap
include_ears: False
include_hair: False
include_glasses: True
[mask.custom]
batch-size: 8
centering: face
fill: False
[mask.unet_dfl]
batch-size: 8
[mask.vgg_clear]
batch-size: 6
[mask.vgg_obstructed]
batch-size: 2
[recognition.vgg_face2]
batch-size: 16
cpu: False
--------- gui.ini ---------
[global]
fullscreen: False
tab: extract
options_panel_width: 30
console_panel_height: 20
icon_size: 14
font: default
font_size: 9
autosave_last_session: prompt
timeout: 120
auto_load_model_stats: True
--------- train.ini ---------
[global]
centering: face
coverage: 87.5
icnr_init: False
conv_aware_init: False
optimizer: adam
learning_rate: 5e-05
epsilon_exponent: -7
save_optimizer: exit
lr_finder_iterations: 1000
lr_finder_mode: set
lr_finder_strength: default
autoclip: False
reflect_padding: False
allow_growth: False
mixed_precision: False
nan_protection: True
convert_batchsize: 16
[global.loss]
loss_function: ssim
loss_function_2: mse
loss_weight_2: 100
loss_function_3: none
loss_weight_3: 0
loss_function_4: none
loss_weight_4: 0
mask_loss_function: mse
eye_multiplier: 3
mouth_multiplier: 2
penalized_mask_loss: True
mask_type: extended
mask_dilation: 0.0
mask_blur_kernel: 3
mask_threshold: 4
learn_mask: False
[model.dfaker]
output_size: 128
[model.dfl_h128]
lowmem: False
[model.dfl_sae]
input_size: 128
architecture: df
autoencoder_dims: 0
encoder_dims: 42
decoder_dims: 21
multiscale_decoder: False
[model.dlight]
features: best
details: good
output_size: 256
[model.original]
lowmem: False
[model.phaze_a]
output_size: 128
shared_fc: None
enable_gblock: True
split_fc: True
split_gblock: False
split_decoders: False
enc_architecture: fs_original
enc_scaling: 7
enc_load_weights: True
bottleneck_type: dense
bottleneck_norm: None
bottleneck_size: 1024
bottleneck_in_encoder: True
fc_depth: 1
fc_min_filters: 1024
fc_max_filters: 1024
fc_dimensions: 4
fc_filter_slope: -0.5
fc_dropout: 0.0
fc_upsampler: upsample2d
fc_upsamples: 1
fc_upsample_filters: 512
fc_gblock_depth: 3
fc_gblock_min_nodes: 512
fc_gblock_max_nodes: 512
fc_gblock_filter_slope: -0.5
fc_gblock_dropout: 0.0
dec_upscale_method: subpixel
dec_upscales_in_fc: 0
dec_norm: None
dec_min_filters: 64
dec_max_filters: 512
dec_slope_mode: full
dec_filter_slope: -0.45
dec_res_blocks: 1
dec_output_kernel: 5
dec_gaussian: True
dec_skip_last_residual: True
freeze_layers: keras_encoder
load_layers: encoder
fs_original_depth: 4
fs_original_min_filters: 128
fs_original_max_filters: 1024
fs_original_use_alt: False
mobilenet_width: 1.0
mobilenet_depth: 1
mobilenet_dropout: 0.001
mobilenet_minimalistic: False
[model.realface]
input_size: 64
output_size: 128
dense_nodes: 1536
complexity_encoder: 128
complexity_decoder: 512
[model.unbalanced]
input_size: 128
lowmem: False
nodes: 1024
complexity_encoder: 128
complexity_decoder_a: 384
complexity_decoder_b: 512
[model.villain]
lowmem: False
[trainer.original]
preview_images: 14
mask_opacity: 30
mask_color: #ff0000
zoom_amount: 5
rotation_range: 10
shift_range: 5
flip_chance: 50
color_lightness: 30
color_ab: 8
color_clahe_chance: 50
color_clahe_max_size: 4