Page 6 of 7
Re: [Guide] Using Faceswap on Nvidia RTX 30xx cards
Posted: Mon Jul 12, 2021 11:44 am
by torzdf
joezi wrote: ↑Mon Jul 12, 2021 2:58 am
torzdf wrote: ↑Mon Jul 12, 2021 12:17 am
Enable "Allow-Growth" in Global Model Settings.
Thanks, but no change. Same errors.
Any news on the official support? It seems conda has added tensorflow 2.5 to their package repo week a go. Updating faceswap to support 2.5 would allow 11.2 cuda/8.2 cudnn combo to work.
edit: manaaged to get it to run once by having some background load on the card when starting training...
edit2: yeah, related to vram. I can get it to run by having something running in the background and then stopping it in the right point when the training process is running so that it wont go OOM.
Yeah, it's VRAM related for sure.
Whilst TF2.5 has been released by Anaconda for Windows, it's still not out for Linux. Once it is, I will test and update.
Re: [Guide] Using Faceswap on Nvidia RTX 30xx cards
Posted: Mon Jul 12, 2021 8:48 pm
by joezi
torzdf wrote: ↑Mon Jul 12, 2021 11:44 am
Yeah, it's VRAM related for sure.
Whilst TF2.5 has been released by Anaconda for Windows, it's still not out for Linux. Once it is, I will test and update.
Ok, good to know.
If anyone else runs into this beofre that, I finally managed to get it to work properly by setting environment variable TF_FORCE_GPU_ALLOW_GROWTH=true. Seems like the tf.config.experimental.set_memory_growth() doesnt work at all with this card and windows.
Re: [Guide] Using Faceswap on Nvidia RTX 30xx cards
Posted: Wed Jul 14, 2021 10:26 am
by babala
so.. is this normal training for 3080 cards?
Phaze-A stojo preset
Mixed precision on
Color Augmentaion - Color ab 15
and nothing touched.
1st training attempt was good when training starts those blank area happend but goin on 150k ite smooth.
next and 3rd is just still shows blank areas to 10k ite or more.
every time I restart program and reboot with doing delete files in model folder.
should I wait for more ite to get mask or somethin to process?
Re: [Guide] Using Faceswap on Nvidia RTX 30xx cards
Posted: Fri Jul 16, 2021 10:01 am
by torzdf
The solid colors mean model corruption. You can try lowering the learning rate. That preset is particularly complex, so can be susceptible to collapse at higher learning rate.
Also, if using mixed precision, I highly recommend setting the Epsilon Exponent to -5 for that preset.
Process Exited No Clear Error Message
Posted: Tue Jul 20, 2021 3:36 am
by Hollywood
Note: I am using a 3000 series card and I know it isn't officially supported but I'm hoping someone may be able to help. I installed using this guide:viewtopic.php?p=5466#p5466
When I try and extract I get this readout...
Code: Select all
Loading...
Setting Faceswap backend to NVIDIA
07/19/2021 23:34:18 INFO Log level set to: DEBUG
2021-07-19 23:34:18.743051: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library cudart64_110.dll
07/19/2021 23:34:20 INFO Output Directory: G:\Deepfake\Frames\Dwight
07/19/2021 23:34:21 VERBOSE Alignments filepath: 'G:\Deepfake\Originals\Dwight\Dwight_alignments.fsa'
07/19/2021 23:34:21 INFO Loading Detect from S3Fd plugin...
07/19/2021 23:34:21 VERBOSE Loading config: 'C:\Users\(removed)\faceswap\config\extract.ini'
07/19/2021 23:34:21 INFO Loading Align from Fan plugin...
07/19/2021 23:34:21 VERBOSE Loading config: 'C:\Users\(removed)\faceswap\config\extract.ini'
07/19/2021 23:34:21 INFO Loading Mask from Components plugin...
07/19/2021 23:34:21 VERBOSE Loading config: 'C:\Users\(removed)\faceswap\config\extract.ini'
07/19/2021 23:34:21 INFO Loading Mask from Extended plugin...
07/19/2021 23:34:21 VERBOSE Loading config: 'C:\Users\(removed)\faceswap\config\extract.ini'
07/19/2021 23:34:21 VERBOSE NVIDIA GeForce RTX 3060 - 10687MB free of 12288MB
07/19/2021 23:34:21 INFO Starting, this may take a while...
07/19/2021 23:34:21 INFO Initializing S3FD (Detect)...
2021-07-19 23:34:21.489190: I tensorflow/compiler/jit/xla_cpu_device.cc:41] Not creating XLA devices, tf_xla_enable_xla_devices not set
2021-07-19 23:34:21.489681: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library nvcuda.dll
2021-07-19 23:34:21.503693: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1720] Found device 0 with properties:
pciBusID: 0000:0a:00.0 name: NVIDIA GeForce RTX 3060 computeCapability: 8.6
coreClock: 1.837GHz coreCount: 28 deviceMemorySize: 12.00GiB deviceMemoryBandwidth: 335.32GiB/s
2021-07-19 23:34:21.503961: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library cudart64_110.dll
2021-07-19 23:34:21.553612: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library cublas64_11.dll
2021-07-19 23:34:21.553780: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library cublasLt64_11.dll
2021-07-19 23:34:21.583953: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library cufft64_10.dll
2021-07-19 23:34:21.590398: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library curand64_10.dll
2021-07-19 23:34:21.620782: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library cusolver64_10.dll
2021-07-19 23:34:21.652701: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library cusparse64_11.dll
2021-07-19 23:34:21.653485: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library cudnn64_8.dll
07/19/2021 23:34:21 INFO Setting allow growth for GPU: PhysicalDevice(name='/physical_device:GPU:0', device_type='GPU')
2021-07-19 23:34:21.653647: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1862] Adding visible gpu devices: 0
2021-07-19 23:34:21.663254: I tensorflow/core/platform/cpu_feature_guard.cc:142] This TensorFlow binary is optimized with oneAPI Deep Neural Network Library (oneDNN) to use the following CPU instructions in performance-critical operations: AVX2
To enable them in other operations, rebuild TensorFlow with the appropriate compiler flags.
2021-07-19 23:34:21.664255: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1720] Found device 0 with properties:
pciBusID: 0000:0a:00.0 name: NVIDIA GeForce RTX 3060 computeCapability: 8.6
coreClock: 1.837GHz coreCount: 28 deviceMemorySize: 12.00GiB deviceMemoryBandwidth: 335.32GiB/s
2021-07-19 23:34:21.664469: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library cudart64_110.dll
2021-07-19 23:34:21.664561: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library cublas64_11.dll
2021-07-19 23:34:21.664645: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library cublasLt64_11.dll
2021-07-19 23:34:21.664720: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library cufft64_10.dll
2021-07-19 23:34:21.664795: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library curand64_10.dll
2021-07-19 23:34:21.664874: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library cusolver64_10.dll
2021-07-19 23:34:21.664951: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library cusparse64_11.dll
2021-07-19 23:34:21.665039: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library cudnn64_8.dll
2021-07-19 23:34:21.665123: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1862] Adding visible gpu devices: 0
2021-07-19 23:34:22.048924: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1261] Device interconnect StreamExecutor with strength 1 edge matrix:
2021-07-19 23:34:22.049094: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1267] 0
2021-07-19 23:34:22.049146: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1280] 0: N
2021-07-19 23:34:22.049321: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1406] Created TensorFlow device (/job:localhost/replica:0/task:0/device:GPU:0 with 10491 MB memory) -> physical GPU (device: 0, name: NVIDIA GeForce RTX 3060, pci bus id: 0000:0a:00.0, compute capability: 8.6)
2021-07-19 23:34:22.049798: I tensorflow/compiler/jit/xla_gpu_device.cc:99] Not creating XLA devices, tf_xla_enable_xla_devices not set
07/19/2021 23:34:22 VERBOSE Initializing plugin model: S3FD
07/19/2021 23:34:22 INFO Initialized S3FD (Detect) with batchsize of 4
07/19/2021 23:34:22 INFO Initializing FAN (Align)...
07/19/2021 23:34:22 INFO Setting allow growth for GPU: PhysicalDevice(name='/physical_device:GPU:0', device_type='GPU')
07/19/2021 23:34:22 VERBOSE Initializing plugin model: FAN
2021-07-19 23:34:23.134322: I tensorflow/compiler/mlir/mlir_graph_optimization_pass.cc:116] None of the MLIR optimization passes are enabled (registered 2)
2021-07-19 23:34:23.649479: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library cudnn64_8.dll
Process exited.
I know it has to do with the card because when I try using the CPU I get a similar readout but it works instead of saying "process exited"
Many thanks in advance
Re: Process Exited No Clear Error Message
Posted: Tue Jul 20, 2021 9:53 am
by torzdf
Hollywood wrote: ↑Tue Jul 20, 2021 3:36 am
I know it has to do with the card because when I try using the CPU I get a similar readout but it works instead of saying "process exited"
Many thanks in advance
Unfortunately that output doesn't show any indication of why it might be exiting. Can you post your system info (output from Help > Output System Information)
Re: Process Exited No Clear Error Message
Posted: Tue Jul 20, 2021 1:50 pm
by Hollywood
torzdf wrote: ↑Tue Jul 20, 2021 9:53 am
Unfortunately that output doesn't show any indication of why it might be exiting. Can you post your system info (output from Help > Output System Information)
Yeah here's the output from that:
Code: Select all
============ System Information ============
encoding: cp1252
git_branch: master
git_commits: 55bb723 New Model: Phaze-A
gpu_cuda: 11.2
gpu_cudnn: No global version found. Check Conda packages for Conda cuDNN
gpu_devices: GPU_0: NVIDIA GeForce RTX 3060
gpu_devices_active: GPU_0
gpu_driver: 471.41
gpu_vram: GPU_0: 12288MB
os_machine: AMD64
os_platform: Windows-10-10.0.22000-SP0
os_release: 10
py_command: C:\Users\(removed)\faceswap/faceswap.py gui
py_conda_version: conda 4.10.3
py_implementation: CPython
py_version: 3.8.10
py_virtual_env: True
sys_cores: 12
sys_processor: AMD64 Family 23 Model 113 Stepping 0, AuthenticAMD
sys_ram: Total: 16329MB, Available: 8518MB, Used: 7811MB, Free: 8518MB
=============== Pip Packages ===============
absl-py==0.13.0
astunparse==1.6.3
brotlipy==0.7.0
cachetools==4.2.2
certifi==2020.6.20
cffi @ file:///C:/ci/cffi_1625831749120/work
charset-normalizer==2.0.3
cryptography @ file:///C:/ci/cryptography_1616769344312/work
cycler==0.10.0
fastcluster==1.1.26
ffmpy==0.2.3
flatbuffers==1.12
gast==0.3.3
google-auth==1.33.0
google-auth-oauthlib==0.4.4
google-pasta==0.2.0
grpcio==1.32.0
h5py==2.10.0
idna @ file:///tmp/build/80754af9/idna_1622654382723/work
imageio @ file:///tmp/build/80754af9/imageio_1617700267927/work
imageio-ffmpeg @ file:///home/conda/feedstock_root/build_artifacts/imageio-ffmpeg_1621542018480/work
joblib @ file:///tmp/build/80754af9/joblib_1613502643832/work
keras-nightly==2.5.0.dev2021032900
Keras-Preprocessing==1.1.2
kiwisolver @ file:///C:/ci/kiwisolver_1612282606037/work
Markdown==3.3.4
matplotlib @ file:///C:/ci/matplotlib-base_1592837548929/work
mkl-fft==1.3.0
mkl-random==1.1.1
mkl-service==2.3.0
numpy @ file:///C:/ci/numpy_and_numpy_base_1603466732592/work
nvidia-ml-py3 @ git+https://github.com/deepfakes/nvidia-ml-py3.git@6fc29ac84b32bad877f078cb4a777c1548a00bf6
oauthlib==3.1.1
olefile==0.46
opencv-python==4.5.3.56
opt-einsum==3.3.0
pathlib==1.0.1
Pillow @ file:///C:/ci/pillow_1625663293593/work
protobuf==3.17.3
psutil @ file:///C:/ci/psutil_1612298324802/work
pyasn1==0.4.8
pyasn1-modules==0.2.8
pycparser @ file:///tmp/build/80754af9/pycparser_1594388511720/work
pyOpenSSL @ file:///tmp/build/80754af9/pyopenssl_1608057966937/work
pyparsing @ file:///home/linux1/recipes/ci/pyparsing_1610983426697/work
PySocks @ file:///C:/ci/pysocks_1605287845585/work
python-dateutil @ file:///tmp/build/80754af9/python-dateutil_1626374649649/work
pywin32==227
requests==2.26.0
requests-oauthlib==1.3.0
rsa==4.7.2
scikit-learn @ file:///C:/ci/scikit-learn_1622739500535/work
scipy @ file:///C:/ci/scipy_1616703433439/work
sip==4.19.13
six==1.15.0
tensorboard==2.5.0
tensorboard-data-server==0.6.1
tensorboard-plugin-wit==1.8.0
tensorflow-estimator==2.4.0
tensorflow-gpu==2.4.1
termcolor==1.1.0
threadpoolctl @ file:///tmp/build/80754af9/threadpoolctl_1626115094421/work
tornado @ file:///C:/ci/tornado_1606942392901/work
tqdm @ file:///tmp/build/80754af9/tqdm_1625563689033/work
typing-extensions==3.7.4.3
urllib3 @ file:///tmp/build/80754af9/urllib3_1603305693037/work
Werkzeug==2.0.1
win-inet-pton @ file:///C:/ci/win_inet_pton_1605306167264/work
wincertstore==0.2
wrapt==1.12.1
============== Conda Packages ==============
# packages in environment at C:\Users\(removed)\MiniConda3\envs\faceswap:
#
# Name Version Build Channel
absl-py 0.13.0 pypi_0 pypi
astunparse 1.6.3 pypi_0 pypi
blas 1.0 mkl
brotli 1.0.9 ha925a31_2
brotlipy 0.7.0 py38h2bbff1b_1003
ca-certificates 2020.10.14 0 anaconda
cachetools 4.2.2 pypi_0 pypi
certifi 2020.6.20 py38_0 anaconda
cffi 1.14.6 py38h2bbff1b_0
charset-normalizer 2.0.3 pypi_0 pypi
cryptography 3.4.7 py38h71e12ea_0
cudatoolkit 10.1.243 h74a9793_0
cudnn 7.6.5 cuda10.1_0
cycler 0.10.0 py38_0
fastcluster 1.1.26 py38h251f6bf_2 conda-forge
ffmpeg 4.2.2 he774522_0
ffmpy 0.2.3 pypi_0 pypi
flatbuffers 1.12 pypi_0 pypi
freetype 2.10.4 hd328e21_0
gast 0.3.3 pypi_0 pypi
git 2.23.0 h6bb4b03_0
google-auth 1.33.0 pypi_0 pypi
google-auth-oauthlib 0.4.4 pypi_0 pypi
google-pasta 0.2.0 pypi_0 pypi
grpcio 1.32.0 pypi_0 pypi
h5py 2.10.0 pypi_0 pypi
icc_rt 2019.0.0 h0cc432a_1
icu 58.2 ha925a31_3
idna 3.2 pyhd3eb1b0_0
imageio 2.9.0 pyhd3eb1b0_0
imageio-ffmpeg 0.4.4 pyhd8ed1ab_0 conda-forge
intel-openmp 2021.3.0 haa95532_3372
joblib 1.0.1 pyhd3eb1b0_0
jpeg 9b hb83a4c4_2
keras-nightly 2.5.0.dev2021032900 pypi_0 pypi
keras-preprocessing 1.1.2 pypi_0 pypi
kiwisolver 1.3.1 py38hd77b12b_0
libpng 1.6.37 h2a8f88b_0
libtiff 4.2.0 hd0e1b90_0
lz4-c 1.9.3 h2bbff1b_0
markdown 3.3.4 pypi_0 pypi
matplotlib 3.2.2 0
matplotlib-base 3.2.2 py38h64f37c6_0
mkl 2020.2 256
mkl-service 2.3.0 py38h196d8e1_0
mkl_fft 1.3.0 py38h46781fe_0
mkl_random 1.1.1 py38h47e9c7a_0
numpy 1.19.2 py38hadc3359_0
numpy-base 1.19.2 py38ha3acd2a_0
nvidia-ml-py3 7.352.1 pypi_0 pypi
oauthlib 3.1.1 pypi_0 pypi
olefile 0.46 py_0
opencv-python 4.5.3.56 pypi_0 pypi
openssl 1.1.1k h2bbff1b_0
opt-einsum 3.3.0 pypi_0 pypi
pathlib 1.0.1 py_1
pillow 8.3.1 py38h4fa10fc_0
pip 21.1.3 py38haa95532_0
protobuf 3.17.3 pypi_0 pypi
psutil 5.8.0 py38h2bbff1b_1
pyasn1 0.4.8 pypi_0 pypi
pyasn1-modules 0.2.8 pypi_0 pypi
pycparser 2.20 py_2
pyopenssl 20.0.1 pyhd3eb1b0_1
pyparsing 2.4.7 pyhd3eb1b0_0
pyqt 5.9.2 py38ha925a31_4
pysocks 1.7.1 py38haa95532_0
python 3.8.10 hdbf39b2_7
python-dateutil 2.8.2 pyhd3eb1b0_0
python_abi 3.8 2_cp38 conda-forge
pywin32 227 py38he774522_1
qt 5.9.7 vc14h73c81de_0
requests 2.26.0 pypi_0 pypi
requests-oauthlib 1.3.0 pypi_0 pypi
rsa 4.7.2 pypi_0 pypi
scikit-learn 0.24.2 py38hf11a4ad_1
scipy 1.6.2 py38h14eb087_0
setuptools 52.0.0 py38haa95532_0
sip 4.19.13 py38ha925a31_0
six 1.15.0 pypi_0 pypi
sqlite 3.36.0 h2bbff1b_0
tensorboard 2.5.0 pypi_0 pypi
tensorboard-data-server 0.6.1 pypi_0 pypi
tensorboard-plugin-wit 1.8.0 pypi_0 pypi
tensorflow-estimator 2.4.0 pypi_0 pypi
tensorflow-gpu 2.4.1 pypi_0 pypi
termcolor 1.1.0 pypi_0 pypi
threadpoolctl 2.2.0 pyhb85f177_0
tk 8.6.10 he774522_0
tornado 6.1 py38h2bbff1b_0
tqdm 4.61.2 pyhd3eb1b0_1
typing-extensions 3.7.4.3 pypi_0 pypi
urllib3 1.25.11 py_0 anaconda
vc 14.2 h21ff451_1
vs2015_runtime 14.27.29016 h5e58377_2
werkzeug 2.0.1 pypi_0 pypi
wheel 0.36.2 pyhd3eb1b0_0
win_inet_pton 1.1.0 py38haa95532_0
wincertstore 0.2 py38_0
wrapt 1.12.1 pypi_0 pypi
xz 5.2.5 h62dcd97_0
zlib 1.2.11 h62dcd97_4
zstd 1.4.9 h19a0ad4_0
================= Configs ==================
--------- .faceswap ---------
backend: nvidia
--------- convert.ini ---------
[color.color_transfer]
clip: True
preserve_paper: True
[color.manual_balance]
colorspace: HSV
balance_1: 0.0
balance_2: 0.0
balance_3: 0.0
contrast: 0.0
brightness: 0.0
[color.match_hist]
threshold: 99.0
[mask.box_blend]
type: gaussian
distance: 11.0
radius: 5.0
passes: 1
[mask.mask_blend]
type: normalized
kernel_size: 3
passes: 4
threshold: 4
erosion: 0.0
[scaling.sharpen]
method: none
amount: 150
radius: 0.3
threshold: 5.0
[writer.ffmpeg]
container: mp4
codec: libx264
crf: 23
preset: medium
tune: none
profile: auto
level: auto
skip_mux: False
[writer.gif]
fps: 25
loop: 0
palettesize: 256
subrectangles: False
[writer.opencv]
format: png
draw_transparent: False
jpg_quality: 75
png_compress_level: 3
[writer.pillow]
format: png
draw_transparent: False
optimize: False
gif_interlace: True
jpg_quality: 75
png_compress_level: 3
tif_compression: tiff_deflate
--------- extract.ini ---------
[global]
allow_growth: True
[align.fan]
batch-size: 12
[detect.cv2_dnn]
confidence: 50
[detect.mtcnn]
minsize: 20
scalefactor: 0.709
batch-size: 8
threshold_1: 0.6
threshold_2: 0.7
threshold_3: 0.7
[detect.s3fd]
confidence: 70
batch-size: 4
[mask.bisenet_fp]
batch-size: 8
include_ears: False
include_hair: False
include_glasses: True
[mask.unet_dfl]
batch-size: 8
[mask.vgg_clear]
batch-size: 6
[mask.vgg_obstructed]
batch-size: 2
--------- gui.ini ---------
[global]
fullscreen: False
tab: extract
options_panel_width: 30
console_panel_height: 20
icon_size: 14
font: default
font_size: 9
autosave_last_session: prompt
timeout: 120
auto_load_model_stats: True
--------- train.ini ---------
[global]
centering: face
coverage: 68.75
icnr_init: False
conv_aware_init: False
optimizer: adam
learning_rate: 5e-05
epsilon_exponent: -7
reflect_padding: False
allow_growth: False
mixed_precision: False
nan_protection: True
convert_batchsize: 16
[global.loss]
loss_function: ssim
mask_loss_function: mse
l2_reg_term: 100
eye_multiplier: 3
mouth_multiplier: 2
penalized_mask_loss: True
mask_type: extended
mask_blur_kernel: 3
mask_threshold: 4
learn_mask: False
[model.dfaker]
output_size: 128
[model.dfl_h128]
lowmem: False
[model.dfl_sae]
input_size: 128
clipnorm: True
architecture: df
autoencoder_dims: 0
encoder_dims: 42
decoder_dims: 21
multiscale_decoder: False
[model.dlight]
features: best
details: good
output_size: 256
[model.original]
lowmem: False
[model.phaze_a]
output_size: 128
shared_fc: none
enable_gblock: True
split_fc: True
split_gblock: False
split_decoders: False
enc_architecture: fs_original
enc_scaling: 40
enc_load_weights: True
bottleneck_type: dense
bottleneck_norm: none
bottleneck_size: 1024
bottleneck_in_encoder: True
fc_depth: 1
fc_min_filters: 1024
fc_max_filters: 1024
fc_dimensions: 4
fc_filter_slope: -0.5
fc_dropout: 0.0
fc_upsampler: upsample2d
fc_upsamples: 1
fc_upsample_filters: 512
fc_gblock_depth: 3
fc_gblock_min_nodes: 512
fc_gblock_max_nodes: 512
fc_gblock_filter_slope: -0.5
fc_gblock_dropout: 0.0
dec_upscale_method: subpixel
dec_norm: none
dec_min_filters: 64
dec_max_filters: 512
dec_filter_slope: -0.45
dec_res_blocks: 1
dec_output_kernel: 5
dec_gaussian: True
dec_skip_last_residual: True
freeze_layers: keras_encoder
load_layers: encoder
fs_original_depth: 4
fs_original_min_filters: 128
fs_original_max_filters: 1024
mobilenet_width: 1.0
mobilenet_depth: 1
mobilenet_dropout: 0.001
[model.realface]
input_size: 64
output_size: 128
dense_nodes: 1536
complexity_encoder: 128
complexity_decoder: 512
[model.unbalanced]
input_size: 128
lowmem: False
clipnorm: True
nodes: 1024
complexity_encoder: 128
complexity_decoder_a: 384
complexity_decoder_b: 512
[model.villain]
lowmem: False
[trainer.original]
preview_images: 14
zoom_amount: 5
rotation_range: 10
shift_range: 5
flip_chance: 50
color_lightness: 30
color_ab: 8
color_clahe_chance: 50
color_clahe_max_size: 4
Re: Process Exited No Clear Error Message
Posted: Tue Jul 20, 2021 9:52 pm
by joezi
Hollywood wrote: ↑Tue Jul 20, 2021 1:50 pm
Yeah here's the output from that:
Code: Select all
gpu_cudnn: No global version found. Check Conda packages for Conda cuDNN
There's atleast part of your problem.
Re: Process Exited No Clear Error Message
Posted: Tue Jul 20, 2021 10:17 pm
by Hollywood
There's atleast part of your problem.
Yeah I was thinking that could be it. I have cudnn 8.1.1 installed but I'll try and reinstall it.
Re: Process Exited No Clear Error Message
Posted: Wed Jul 21, 2021 3:42 am
by Hollywood
Yep that fixed it. I manually added the cudNN libs into the faceswap environment and it works now. Thanks for the help!
Re: [Guide] Using Faceswap on Nvidia RTX 30xx cards
Posted: Wed Sep 08, 2021 4:52 pm
by Faceswappnooob
I can confirm that the automated install now works with RTX 3080 ROG STRIX Card.
I needed to uninstall MiniPython and perform install to a new folder, it gave a warning when trying to overwrite existing install.
I was able to continue training with existing model .
Upgrade from a 2070 at 1940 iterations in 5 mins to
2080 at 2476 iterations.
Extration of a sample video was also about 40% faster
Re: [Guide] Using Faceswap on Nvidia RTX 30xx cards
Posted: Thu Sep 09, 2021 12:05 am
by torzdf
Thanks for the feedback, that is useful information
Re: [Guide] Using Faceswap on Nvidia RTX 30xx cards
Posted: Thu Sep 23, 2021 6:48 pm
by paruru715
The new version works right away with RTX 3070.
There are two issues however,
- It won't pick up where I left off from an old project:
Code: Select all
Loading...
Setting Faceswap backend to NVIDIA
09/24/2021 02:39:40 INFO Log level set to: INFO
09/24/2021 02:39:43 ERROR The input folder '[Redacted FACE A directory here]' contains images that are not extracted faces.
09/24/2021 02:39:43 ERROR You can only train a model on faces generated from Faceswap's extract process. Please check your sources and try again.
Process exited.
Seems like it can't recognize extracted face from an older version of Faceswap.
- At 30,000 iterations, on Dlight, the Swap side of the preview is not developing at all. Not sure if it is a Dlight problem or compatibility/corruption problem that has to do with the RTX 3070.
The Swap (B) side for the Swap > Original is not developing as shown below. However, everything else looks as expected.
Re: [Guide] Using Faceswap on Nvidia RTX 30xx cards
Posted: Sat Sep 25, 2021 2:36 am
by babala
I've been using app 2 month without any problems now, thanks to Mods.
I just checked update that support 30xx cards.
should I remove current apps and update for better performence?
like I got training Phaze-A 12.8 EGs/sec now, update and then go to 20 EGs/sec maybe?
edit : so I re-install latest vesion app and work find. I got an 'custom mask etc' error and checked viewtopic.php?f=6&t=1744 this thread.
but 25% ish slower now.
batch 4 Phaze-A 12.5 EGs/sec back then, now batch 4 Phaze-A 9.0 EGs/sec.
I think TF 2.6 is the problem, acording that thread.
should I downgrades TF 2.6 to 2.4 ?
Re: [Guide] Using Faceswap on Nvidia RTX 30xx cards
Posted: Mon Sep 27, 2021 9:06 am
by torzdf
paruru715 wrote: ↑Thu Sep 23, 2021 6:48 pm
The new version works right away with RTX 3070.
There are two issues however,
- It won't pick up where I left off from an old project:
Code: Select all
Loading...
Setting Faceswap backend to NVIDIA
09/24/2021 02:39:40 INFO Log level set to: INFO
09/24/2021 02:39:43 ERROR The input folder '[Redacted FACE A directory here]' contains images that are not extracted faces.
09/24/2021 02:39:43 ERROR You can only train a model on faces generated from Faceswap's extract process. Please check your sources and try again.
Process exited.
Seems like it can't recognize extracted face from an older version of Faceswap.
Yeah, your last version must be super old. Putting alignment information into PNG headers was something implemented a long time ago. We did have an interim period when facesets got auto-upgraded, but sadly no help to you as that time has long since past
- At 30,000 iterations, on Dlight, the Swap side of the preview is not developing at all. Not sure if it is a Dlight problem or compatibility/corruption problem that has to do with the RTX 3070.
The Swap (B) side for the Swap > Original is not developing as shown below. However, everything else looks as expected.
That may be how dlight likes to work. I can't remember to be honest, but it is unbalanced towards the B side, so it seems possible.
Re: [Guide] Using Faceswap on Nvidia RTX 30xx cards
Posted: Mon Sep 27, 2021 9:08 am
by torzdf
babala wrote: ↑Sat Sep 25, 2021 2:36 am
I think TF 2.6 is the problem, acording that thread.
should I downgrades TF 2.6 to 2.4 ?
Whichever is easiest for you to be honest. If you have a 30xx card, then you will need to stay on 2.6, otherwise downgrading to 2,4 is fine as it is fully supported for earlier gpus
Re: [Guide] Using Faceswap on Nvidia RTX 30xx cards
Posted: Mon Sep 27, 2021 9:42 am
by babala
torzdf wrote: ↑Mon Sep 27, 2021 9:08 am
Whichever is easiest for you to be honest. If you have a 30xx card, then you will need to stay on 2.6, otherwise downgrading to 2,4 is fine as it is fully supported for earlier gpus
tried re-install manually several times of TF-CPU and TF-GPU ways, but ends up with no detect GPU problem one.
and your latest methods too, so
I'll just stick with TF 2.6 now for any updates. 25% slower tho
I can still training, thank you so much
Re: [Guide] Using Faceswap on Nvidia RTX 30xx cards
Posted: Mon Oct 04, 2021 5:16 pm
by Barnuble
Hi Everybody,
I own an RTX 3090.
I reinstalled FaceSwap recently with the new installer and saw about a 12% slowdown (with TensorFlow 2.6).
I have just managed to switch back to version 2.4 and I found the performance that I had before.
For those interested, here's how I did it :
First, I followed the guide from torzdf :
conda env remove -n faceswap
conda create -n faceswap python=3.8
conda activate faceswap
cd faceswap
conda install git
git fetch --unshallow
git checkout c7d85f89e69c74e97bf7485b064c07487d31faae
python setup.py
I restarted FaceSwap but training was not working anymore... I decided to reinstall once TF 2.4 in this way :
conda activate faceswap
conda remove tensorflow
conda install brotli
conda install urllib3
conda install -c anaconda urllib3
pip install tensorflow-gpu==2.4.1
And now, it works like before without error message !!!
For information (Testing my two cards : RTX 3090 / RTX 2080Ti) :
RTX 3090 - TF 2.6 / RealFace @Batch 256 (Mixed-Precision) = 364.5 Egs/s
RTX 3090 - TF 2.4 / RealFace @Batch 256 (Mixed-Precision) = 410.2 Egs/s
RTX 2080Ti - TF 2.4 / RealFace @Batch 96 (Mixed-Precision) = 257.4 Egs/s
I am very happy and I hope this will be usefull for someone
Re: [Guide] Using Faceswap on Nvidia RTX 30xx cards
Posted: Thu Oct 07, 2021 6:54 am
by babala
Barnuble wrote: ↑Mon Oct 04, 2021 5:16 pm
I own an RTX 3090.
I reinstalled FaceSwap recently with the new installer and saw about a 12% slowdown (with TensorFlow 2.6).
I have just managed to switch back to version 2.4 and I found the performance that I had before.
in my case, I've got cuda 11.2 and cudnn 8.1.1.33 for TF 2.4.1 not support rtx 30xx version of FaceSwap app.
I saw your guide and followed it, neither guides not working with some error when training
Error in thread (_training_0): ('Keyword argument not understood:', 'keepdims')
or
Not found: No algorithm worked!
so are you installed global cuda and cudnn for tensorflow 2.4 ?
Re: [Guide] Using Faceswap on Nvidia RTX 30xx cards
Posted: Sat Oct 09, 2021 8:21 am
by Barnuble
Here's my System Information (FaceSwap/Help/Output System Information) :
============ System Information ============
...
gpu_cuda: 11.4
gpu_cudnn: 8.2.4
..
gpu_driver: 472.12
...
py_conda_version: conda 4.10.3
...
py_version: 3.8.11
=============== Pip Packages ===============
...
tensorboard==2.6.0
tensorboard-data-server==0.6.1
tensorboard-plugin-wit==1.8.0
tensorflow-estimator==2.4.0
tensorflow-gpu==2.4.1
...
Install Cuda 11.4 and Cudnn 8.2.4
(Restart your computer...)
Start : Windows Menu/Anaconda3/Anaconda Prompt
Force TF-GPU to 2.4.1 installation in PIP Packages using the following commands :
conda activate faceswap
conda remove tensorflow
conda install brotli
conda install urllib3
conda install -c anaconda urllib3
pip install tensorflow-gpu==2.4.1
Exit Anaconda prompt and Restart FaceSwap...
Everything should work !
If it still does not works :
- Uninstall FaceSwap
- Uninstall Cuda and Cudnn
- Clean registry keys using softwares like "CCleaner"
- Restart Computer
- Install Cuda 11.4 and Cudnn 8.2.4
- Install FaceSwap
- Force TF-GPU to 2.4.1 using the guide described above (Conda activate...., .....)
Hope it will help you...