NVIDIA backend is set, but GPU is not used

Installing and setting up FaceSwap


Forum rules

Read the FAQs and search the forum before posting a new topic.

Please mark any answers that fixed your problems so others can find the solutions.

Locked
User avatar
Buffle
Posts: 2
Joined: Sat Oct 24, 2020 12:21 pm

NVIDIA backend is set, but GPU is not used

Post by Buffle »

Hi,
I have installed the faceswap on my Linux Mint install using the installer which automatically created the conda environment and did the python install. I selected (1) for NVIDIA backend. When I launch faceswap, it tells me that NVIDIA backend is used.
However, training is as long as in CPU mode, and if I check my devices usages, I can see that all my CPU cores are running with 100% of usage, but my GPU is not (both usage (around 10%) and allocated memory (around 200M) stay the same). If I do a nvidia-smi during training, I can't see a faceswap process running:

+-----------------------------------------------------------------------------+
| NVIDIA-SMI 455.23.05 Driver Version: 455.23.05 CUDA Version: 11.1 |
|-------------------------------+----------------------+----------------------+
| GPU Name Persistence-M| Bus-Id Disp.A | Volatile Uncorr. ECC |
| Fan Temp Perf Pwr:Usage/Cap| Memory-Usage | GPU-Util Compute M. |
| | | MIG M. |
|===============================+======================+======================|
| 0 GeForce MX110 On | 00000000:01:00.0 Off | N/A |
| N/A 49C P0 N/A / N/A | 243MiB / 2004MiB | 11% Default |
| | | N/A |
+-------------------------------+----------------------+----------------------+


+-----------------------------------------------------------------------------+
| Processes: |
| GPU GI CI PID Type Process name GPU Memory |
| ID ID Usage |
|=============================================================================|
| 0 N/A N/A 921 G /usr/lib/xorg/Xorg 129MiB |
| 0 N/A N/A 1389 G cinnamon 45MiB |
| 0 N/A N/A 2160 G ...AAAAAAAAA= --shared-files 60MiB |
| 0 N/A N/A 5909 G nvidia-settings 0MiB |
+-----------------------------------------------------------------------------+

I've already tried to remove everything and reinstall, but it is still the same problem: no error appears, the program tells me that it is using NVIDIA mode, but computation seems to be done by my CPU cores.
Here is what I get when going to "Output system information" in the GUI:

============ System Information ============
encoding: UTF-8
git_branch: master
git_commits: 42d10bc Typo Fix
gpu_cuda: 9.1
gpu_cudnn: 7.6.5
gpu_devices: GPU_0: GeForce MX110
gpu_devices_active: GPU_0
gpu_driver: 455.23.05
gpu_vram: GPU_0: 2004MB
os_machine: x86_64
os_platform: Linux-4.15.0-117-generic-x86_64-with-glibc2.10
os_release: 4.15.0-117-generic
py_command: faceswap.py gui
py_conda_version: conda 4.8.3
py_implementation: CPython
py_version: 3.8.5
py_virtual_env: True
sys_cores: 4
sys_processor: x86_64
sys_ram: Total: 7859MB, Available: 5173MB, Used: 2251MB, Free: 3256MB

=============== Pip Packages ===============
absl-py @ file:///tmp/build/80754af9/absl-py_1600297518631/work
aiohttp @ file:///tmp/build/80754af9/aiohttp_1602530294624/work
astunparse==1.6.3
async-timeout==3.0.1
attrs @ file:///tmp/build/80754af9/attrs_1600298409949/work
blinker==1.4
brotlipy==0.7.0
cachetools @ file:///tmp/build/80754af9/cachetools_1596822027882/work
certifi==2020.6.20
cffi @ file:///tmp/build/80754af9/cffi_1600699146221/work
chardet==3.0.4
click==7.1.2
cryptography @ file:///tmp/build/80754af9/cryptography_1601046815590/work
cycler==0.10.0
fastcluster==1.1.26
ffmpy==0.2.3
gast==0.3.3
google-auth @ file:///tmp/build/80754af9/google-auth_1601995530934/work
google-auth-oauthlib==0.4.1
google-pasta==0.2.0
grpcio @ file:///tmp/build/80754af9/grpcio_1597424474635/work
h5py @ file:///tmp/build/80754af9/h5py_1593454122442/work
idna @ file:///tmp/build/80754af9/idna_1593446292537/work
imageio @ file:///tmp/build/80754af9/imageio_1594161405741/work
imageio-ffmpeg @ file:///home/conda/feedstock_root/build_artifacts/imageio-ffmpeg_1589202782679/work
importlib-metadata @ file:///tmp/build/80754af9/importlib-metadata_1602276842396/work
joblib @ file:///tmp/build/80754af9/joblib_1601912903842/work
Keras-Preprocessing==1.1.0
kiwisolver==1.2.0
Markdown @ file:///tmp/build/80754af9/markdown_1603216487943/work
matplotlib @ file:///tmp/build/80754af9/matplotlib-base_1592846008246/work
mkl-fft==1.2.0
mkl-random==1.1.1
mkl-service==2.3.0
multidict @ file:///tmp/build/80754af9/multidict_1600456399709/work
numpy @ file:///tmp/build/80754af9/numpy_and_numpy_base_1596233721170/work
nvidia-ml-py3 @ git+https://github.com/deepfakes/nvidia-ml- ... 1548a00bf6
oauthlib==3.1.0
olefile==0.46
opencv-python==4.4.0.44
opt-einsum==3.1.0
pathlib==1.0.1
Pillow @ file:///tmp/build/80754af9/pillow_1602788711040/work
protobuf==3.13.0
psutil @ file:///tmp/build/80754af9/psutil_1598370257551/work
pyasn1==0.4.8
pyasn1-modules==0.2.8
pycparser @ file:///tmp/build/80754af9/pycparser_1594388511720/work
PyJWT==1.7.1
pyOpenSSL @ file:///tmp/build/80754af9/pyopenssl_1594392929924/work
pyparsing==2.4.7
PySocks==1.7.1
python-dateutil==2.8.1
requests @ file:///tmp/build/80754af9/requests_1592841827918/work
requests-oauthlib==1.3.0
rsa @ file:///tmp/build/80754af9/rsa_1596998415516/work
scikit-learn @ file:///tmp/build/80754af9/scikit-learn_1598376899566/work
scipy @ file:///tmp/build/80754af9/scipy_1597686649129/work
sip==4.19.13
six==1.15.0
tensorboard==2.2.1
tensorboard-plugin-wit==1.6.0
tensorflow==2.2.0
tensorflow-estimator==2.2.0
termcolor==1.1.0
threadpoolctl @ file:///tmp/tmp9twdgx9k/threadpoolctl-2.1.0-py3-none-any.whl
tornado==6.0.4
tqdm @ file:///tmp/build/80754af9/tqdm_1602185206534/work
urllib3 @ file:///tmp/build/80754af9/urllib3_1603305693037/work
Werkzeug==1.0.1
wrapt==1.12.1
yarl @ file:///tmp/build/80754af9/yarl_1602693007734/work
zipp @ file:///tmp/build/80754af9/zipp_1603306325174/work

============== Conda Packages ==============

packages in environment at /home/ajuvenn/miniconda3/envs/faceswap:

#

Name Version Build Channel

_libgcc_mutex 0.1 main
_tflow_select 2.1.0 gpu
absl-py 0.10.0 py38_0
aiohttp 3.6.3 py38h7b6447c_0
astunparse 1.6.3 py_0
async-timeout 3.0.1 py38_0
attrs 20.2.0 py_0
blas 1.0 mkl
blinker 1.4 py38_0
brotlipy 0.7.0 py38h7b6447c_1000
bzip2 1.0.8 h516909a_3 conda-forge
c-ares 1.16.1 h7b6447c_0
ca-certificates 2020.10.14 0
cachetools 4.1.1 py_0
certifi 2020.6.20 py38_0
cffi 1.14.3 py38he30daa8_0
chardet 3.0.4 py38_1003
click 7.1.2 py_0
cryptography 3.1.1 py38h1ba5d50_0
cudatoolkit 10.1.243 h6bb024c_0
cudnn 7.6.5 cuda10.1_0
cupti 10.1.168 0
cycler 0.10.0 py38_0
dbus 1.13.18 hb2f20db_0
expat 2.2.10 he6710b0_2
fastcluster 1.1.26 py38hc5bc63f_2 conda-forge
ffmpeg 4.3.1 h3215721_1 conda-forge
ffmpy 0.2.3 pypi_0 pypi
fontconfig 2.13.0 h9420a91_0
freetype 2.10.4 h5ab3b9f_0
gast 0.3.3 py_0
git 2.23.0 pl526hacde149_0
glib 2.66.1 h92f7085_0
gmp 6.2.0 he1b5a44_3 conda-forge
gnutls 3.6.13 h79a8f9a_0 conda-forge
google-auth 1.22.1 py_0
google-auth-oauthlib 0.4.1 py_2
google-pasta 0.2.0 py_0
grpcio 1.31.0 py38hf8bcb03_0
gst-plugins-base 1.14.0 hbbd80ab_1
gstreamer 1.14.0 hb31296c_0
h5py 2.10.0 py38hd6299e0_1
hdf5 1.10.6 hb1b8bf9_0
icu 58.2 he6710b0_3
idna 2.10 py_0
imageio 2.9.0 py_0
imageio-ffmpeg 0.4.2 py_0 conda-forge
importlib-metadata 2.0.0 py_1
intel-openmp 2020.2 254
joblib 0.17.0 py_0
jpeg 9b h024ee3a_2
keras-preprocessing 1.1.0 py_1
kiwisolver 1.2.0 py38hfd86e86_0
krb5 1.18.2 h173b8e3_0
lame 3.100 h14c3975_1001 conda-forge
lcms2 2.11 h396b838_0
ld_impl_linux-64 2.33.1 h53a641e_7
libcurl 7.71.1 h20c2e04_1
libedit 3.1.20191231 h14c3975_1
libffi 3.3 he6710b0_2
libgcc-ng 9.1.0 hdf63c60_0
libgfortran-ng 7.3.0 hdf63c60_0
libiconv 1.16 h516909a_0 conda-forge
libpng 1.6.37 hbc83047_0
libprotobuf 3.13.0.1 hd408876_0
libssh2 1.9.0 h1ba5d50_1
libstdcxx-ng 9.1.0 hdf63c60_0
libtiff 4.1.0 h2733197_1
libuuid 1.0.3 h1bed415_2
libxcb 1.14 h7b6447c_0
libxml2 2.9.10 hb55368b_3
lz4-c 1.9.2 heb0550a_3
markdown 3.3.2 py38_0
matplotlib 3.2.2 0
matplotlib-base 3.2.2 py38hef1b27d_0
mkl 2020.2 256
mkl-service 2.3.0 py38he904b0f_0
mkl_fft 1.2.0 py38h23d657b_0
mkl_random 1.1.1 py38h0573a6f_0
multidict 4.7.6 py38h7b6447c_1
ncurses 6.2 he6710b0_1
nettle 3.4.1 h1bed415_1002 conda-forge
numpy 1.19.1 py38hbc911f0_0
numpy-base 1.19.1 py38hfa32c7d_0
nvidia-ml-py3 7.352.1 pypi_0 pypi
oauthlib 3.1.0 py_0
olefile 0.46 py_0
opencv-python 4.4.0.44 pypi_0 pypi
openh264 2.1.1 h8b12597_0 conda-forge
openssl 1.1.1h h7b6447c_0
opt_einsum 3.1.0 py_0
pathlib 1.0.1 py_1
pcre 8.44 he6710b0_0
perl 5.26.2 h14c3975_0
pillow 8.0.0 py38h9a89aac_0
pip 20.2.4 py38_0
protobuf 3.13.0.1 py38he6710b0_1
psutil 5.7.2 py38h7b6447c_0
pyasn1 0.4.8 py_0
pyasn1-modules 0.2.8 py_0
pycparser 2.20 py_2
pyjwt 1.7.1 py38_0
pyopenssl 19.1.0 py_1
pyparsing 2.4.7 py_0
pyqt 5.9.2 py38h05f1152_4
pysocks 1.7.1 py38_0
python 3.8.5 h7579374_1
python-dateutil 2.8.1 py_0
python_abi 3.8 1_cp38 conda-forge
qt 5.9.7 h5867ecd_1
readline 8.0 h7b6447c_0
requests 2.24.0 py_0
requests-oauthlib 1.3.0 py_0
rsa 4.6 py_0
scikit-learn 0.23.2 py38h0573a6f_0
scipy 1.5.2 py38h0b6359f_0
setuptools 50.3.0 py38hb0f4dca_1
sip 4.19.13 py38he6710b0_0
six 1.15.0 py_0
sqlite 3.33.0 h62c20be_0
tensorboard 2.2.1 pyh532a8cf_0
tensorboard-plugin-wit 1.6.0 py_0
tensorflow 2.2.0 gpu_py38hb782248_0
tensorflow-base 2.2.0 gpu_py38h83e3d50_0
tensorflow-estimator 2.2.0 pyh208ff02_0
tensorflow-gpu 2.2.0 h0d30ee6_0
termcolor 1.1.0 py38_1
threadpoolctl 2.1.0 pyh5ca1d4c_0
tk 8.6.10 hbc83047_0
tornado 6.0.4 py38h7b6447c_1
tqdm 4.50.2 py_0
urllib3 1.25.11 py_0
werkzeug 1.0.1 py_0
wheel 0.35.1 py_0
wrapt 1.12.1 py38h7b6447c_1
x264 1!152.20180806 h14c3975_0 conda-forge
xz 5.2.5 h7b6447c_0
yarl 1.6.2 py38h7b6447c_0
zipp 3.3.1 py_0
zlib 1.2.11 h7b6447c_3
zstd 1.4.5 h9ceee32_0

================= Configs ==================
--------- gui.ini ---------

[global]
fullscreen: False
tab: extract
options_panel_width: 30
console_panel_height: 20
icon_size: 14
font: default
font_size: 9
autosave_last_session: prompt
timeout: 120
auto_load_model_stats: True

--------- .faceswap ---------
backend: nvidia

--------- train.ini ---------

[global]
coverage: 68.75
icnr_init: False
conv_aware_init: False
optimizer: adam
learning_rate: 5e-05
reflect_padding: False
allow_growth: False
mixed_precision: False
convert_batchsize: 16

[global.loss]
loss_function: ssim
mask_loss_function: mse
l2_reg_term: 100
eye_multiplier: 3
mouth_multiplier: 2
penalized_mask_loss: True
mask_type: extended
mask_blur_kernel: 3
mask_threshold: 4
learn_mask: False

[model.dfl_sae]
input_size: 128
clipnorm: True
architecture: df
autoencoder_dims: 0
encoder_dims: 42
decoder_dims: 21
multiscale_decoder: False

[model.dlight]
features: best
details: good
output_size: 256

[model.realface]
input_size: 64
output_size: 128
dense_nodes: 1536
complexity_encoder: 128
complexity_decoder: 512

[model.villain]
lowmem: False

[model.original]
lowmem: False

[model.unbalanced]
input_size: 128
lowmem: False
clipnorm: True
nodes: 1024
complexity_encoder: 128
complexity_decoder_a: 384
complexity_decoder_b: 512

[model.dfl_h128]
lowmem: False

[trainer.original]
preview_images: 14
zoom_amount: 5
rotation_range: 10
shift_range: 5
flip_chance: 50
disable_warp: False
color_lightness: 30
color_ab: 8
color_clahe_chance: 50
color_clahe_max_size: 4

--------- convert.ini ---------

[mask.mask_blend]
type: normalized
kernel_size: 3
passes: 4
threshold: 4
erosion: 0.0

[mask.box_blend]
type: gaussian
distance: 11.0
radius: 5.0
passes: 1

[writer.pillow]
format: png
draw_transparent: False
optimize: False
gif_interlace: True
jpg_quality: 75
png_compress_level: 3
tif_compression: tiff_deflate

[writer.opencv]
format: png
draw_transparent: False
jpg_quality: 75
png_compress_level: 3

[writer.gif]
fps: 25
loop: 0
palettesize: 256
subrectangles: False

[writer.ffmpeg]
container: mp4
codec: libx264
crf: 23
preset: medium
tune: none
profile: auto
level: auto
skip_mux: False

[scaling.sharpen]
method: none
amount: 150
radius: 0.3
threshold: 5.0

[color.match_hist]
threshold: 99.0

[color.manual_balance]
colorspace: HSV
balance_1: 0.0
balance_2: 0.0
balance_3: 0.0
contrast: 0.0
brightness: 0.0

[color.color_transfer]
clip: True
preserve_paper: True

--------- extract.ini ---------

[global]
allow_growth: False

[mask.vgg_obstructed]
batch-size: 2

[mask.vgg_clear]
batch-size: 6

[mask.unet_dfl]
batch-size: 8

[detect.s3fd]
confidence: 70
batch-size: 4

[detect.mtcnn]
minsize: 20
threshold_1: 0.6
threshold_2: 0.7
threshold_3: 0.7
scalefactor: 0.709
batch-size: 8

[detect.cv2_dnn]
confidence: 50

[align.fan]
batch-size: 12

Do you have an idea on how to fix this ?
Thanks !

User avatar
torzdf
Posts: 2671
Joined: Fri Jul 12, 2019 12:53 am
Answers: 159
Has thanked: 131 times
Been thanked: 625 times

Re: NVIDIA backend is set, but GPU is not used

Post by torzdf »

This shouldn't happen on Linux, however, there are always exceptions :(

I would say your system installed Cuda 9.1 is interfering with the Conda installed 10.1 version. Try removing your system Cuda and cuDNN and hopefully it will resolve your issue.

My word is final

User avatar
Buffle
Posts: 2
Joined: Sat Oct 24, 2020 12:21 pm

Re: NVIDIA backend is set, but GPU is not used

Post by Buffle »

Thank you for the answer
I indeed had already an installed cuda version.
So I did a "apt autoremove cuda", deleted my faceswap and conda folders, and launched again the installer. But the problem is still the same. I'll try on windows, maybe it is the more simple.

Locked