Page 1 of 2
"NVML Shared Library Not Found" with new drivers.
Posted: Mon Jul 15, 2019 9:01 pm
by ChilledGumbo
I have updated my drivers to the newest version in NVIDIA GeForce Experience, but there's this NVML shared library issue so I can't even extract images...
I have looked for nvidia-smi.exe in "C:\Program Files\NVIDIA Corporation\NVSMI", but it turns out I never even had the "\NVSMI" folder in the first place.
I am rather confused as to how I could have this error with fresh drivers from GeForce Experience...
Please help...
Re: "NVML Shared Library Not Found" with new drivers.
Posted: Mon Jul 15, 2019 9:15 pm
by torzdf
The most likely issue for this is that you have Windows drivers installed through Windows Update/Windows Store.
This is fine for gaming, but unfortunately presents problems for Machine Learning. Unfortunately Windows does not make it easy to remove this driver and not have it re-install itself straight away.
Obtain the drivers for your card from the Nvidia website. The latest Game Ready drivers will be fine, however make sure you download the standard drivers NOT the DCH drivers
Download Display Driver Uninstaller (DDU)
Disconnect from the internet. This is extremely important!
Remove your current drivers with DDU. Install your new drivers that you downloaded from the Nvidia site.
You should now be good to go.
If this solves your issue, please mark this post as solved.
Re: "NVML Shared Library Not Found" with new drivers.
Posted: Mon Jul 15, 2019 10:08 pm
by ChilledGumbo
Faceswap does not detect my GPU
Posted: Wed Jul 17, 2019 2:15 pm
by erkeis
In my system, the app does not find my GPU&driver.
Though undetectable, my GPU's cuda cores are works and runs extracting or training.
I'd like to know whether this situation is normal.. 
I'm totally noob at these kind of.....haha 
Code: Select all
============ System Information ============
encoding: cp949
git_branch: Not Found
git_commits: Not Found
gpu_cuda: 10.0
gpu_cudnn: 7.3.0
gpu_devices:
gpu_devices_active:
gpu_driver: No Nvidia driver found
gpu_vram:
os_machine: AMD64
os_platform: Windows-10-10.0.18362-SP0
os_release: 10
py_command: C:\Users\kg908\faceswap/faceswap.py gui
py_conda_version: conda 4.7.5
py_implementation: CPython
py_version: 3.6.8
py_virtual_env: True
sys_cores: 16
sys_processor: AMD64 Family 23 Model 8 Stepping 2, AuthenticAMD
sys_ram: Total: 32700MB, Available: 22632MB, Used: 10067MB, Free: 22632MB
Re: Is it necessary detecting GPU&driver for use?
Posted: Wed Jul 17, 2019 3:41 pm
by torzdf
That isn't normal, no. In theory if it can't find your GPU it shouldn't be able to use it (for extraction at least).
Normally I'd try to diagnose this issue, but if it works, then I'd leave it for now until/unless you have problems.
One thing though, could you tell me if nvidia-smi.exe exists in either of the following folders, and if so, which ones:
C:\Program Files\NVIDIA Corporation\NVSMI
C:\Windows\System32
Re: Is it necessary detecting GPU&driver for use?
Posted: Thu Jul 18, 2019 6:43 am
by erkeis
Hi. first of all, Thanks for your reply. 
nvidia-smi.exe is in this path
C:\Windows\System32
In my case, I have no directory "NVSMI" in
C:\Program Files\NVIDIA Corporation\
I tried extracting a video again.
And i found it occupying GPU memory 7.GB and using cuda 80%~.
My graphics card is RTX 2080, (I don't know it might help solve this issue.)
Here's my logs, while [Extract]
Code: Select all
Loading...
07/18/2019 15:20:02 INFO Log level set to: INFO
07/18/2019 15:20:04 INFO Output Directory: Y:\002.Documents\Learning\src\Sbd
07/18/2019 15:20:04 INFO Input Video: Y:\002.Documents\Learning\src\Eunji\non.mp4
07/18/2019 15:20:04 INFO Loading Detect from Mtcnn plugin...
07/18/2019 15:20:04 INFO Loading Align from Fan plugin...
07/18/2019 15:20:04 WARNING There was an error reading from the Nvidia Machine Learning Library. Either you do not have an Nvidia GPU (in which case this warning can be ignored) or the most likely cause is incorrectly installed drivers. If this is the case, Please remove and reinstall your Nvidia drivers before reporting.Original Error: NVML Shared Library Not Found
07/18/2019 15:20:04 WARNING No GPU detected. Switching to CPU mode
07/18/2019 15:20:04 INFO Starting, this may take a while...
07/18/2019 15:20:06 INFO Initializing Face Alignment Network...
07/18/2019 15:20:06 WARNING There was an error reading from the Nvidia Machine Learning Library. Either you do not have an Nvidia GPU (in which case this warning can be ignored) or the most likely cause is incorrectly installed drivers. If this is the case, Please remove and reinstall your Nvidia drivers before reporting.Original Error: NVML Shared Library Not Found
07/18/2019 15:20:06 WARNING No GPU detected. Switching to CPU mode
2019-07-18 15:20:16.750339: E tensorflow/stream_executor/cuda/cuda_driver.cc:806] failed to allocate 8.00G (8589934592 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory
2019-07-18 15:20:16.750614: E tensorflow/stream_executor/cuda/cuda_driver.cc:806] failed to allocate 7.20G (7730940928 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory
2019-07-18 15:20:16.750738: E tensorflow/stream_executor/cuda/cuda_driver.cc:806] failed to allocate 6.48G (6957846528 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory
07/18/2019 15:20:18 INFO Initialized Face Alignment Network.
07/18/2019 15:20:20 INFO Initializing MTCNN Detector...
07/18/2019 15:20:20 WARNING There was an error reading from the Nvidia Machine Learning Library. Either you do not have an Nvidia GPU (in which case this warning can be ignored) or the most likely cause is incorrectly installed drivers. If this is the case, Please remove and reinstall your Nvidia drivers before reporting.Original Error: NVML Shared Library Not Found
07/18/2019 15:20:20 WARNING No GPU detected. Switching to CPU mode
07/18/2019 15:20:21 WARNING From C:\Users\kg908\Anaconda3\envs\faceswap\lib\site-packages\tensorflow\python\framework\op_def_library.py:263: colocate_with (from tensorflow.python.framework.ops) is deprecated and will be removed in a future version.\nInstructions for updating:\nColocations handled automatically by placer.
07/18/2019 15:20:21 WARNING From C:\Users\kg908\faceswap\plugins\extract\detect\mtcnn.py:423: div (from tensorflow.python.ops.math_ops) is deprecated and will be removed in a future version.\nInstructions for updating:\nDeprecated in favor of operator or tf.math.divide.
Exception in Tkinter callback
Traceback (most recent call last):
File "C:\Users\kg908\Anaconda3\envs\faceswap\lib\tkinter\__init__.py", line 1705, in __call__
return self.func(*args)
File "C:\Users\kg908\Anaconda3\envs\faceswap\lib\tkinter\__init__.py", line 749, in callit
func(*args)
File "C:\Users\kg908\faceswap\lib\gui\display_page.py", line 248, in <lambda>
self.after(waittime, lambda t=waittime: self.update_page(t))
File "C:\Users\kg908\faceswap\lib\gui\display_page.py", line 246, in update_page
self.display_item_set()
File "C:\Users\kg908\faceswap\lib\gui\display_command.py", line 28, in display_item_set
frame_dims=(self.winfo_width(), self.winfo_height()))
File "C:\Users\kg908\faceswap\lib\gui\utils.py", line 322, in load_latest_preview
show_image = self.place_previews(frame_dims)
File "C:\Users\kg908\faceswap\lib\gui\utils.py", line 435, in place_previews
for row in range(rows)])
File "C:\Users\kg908\faceswap\lib\gui\utils.py", line 435, in <listcomp>
for row in range(rows)])
File "C:\Users\kg908\Anaconda3\envs\faceswap\lib\site-packages\numpy\core\shape_base.py", line 340, in hstack
return _nx.concatenate(arrs, 1)
ValueError: need at least one array to concatenate
Re: Is it necessary detecting GPU&driver for use?
Posted: Thu Jul 18, 2019 8:19 am
by torzdf
Thanks for the update. This is useful information.
There is currently an issue where if Windows Update/Store installs your Nvidia drivers, then FaceSwap can't read the information.
Extract will fail in this situation as we cannot dynamically allocate VRAM. If you enable "single process" mode you should be able to run extract.
I am going to look at potential solutions our end, as a lot of users will have this kind of setup.
You can try to resolve this yourself by replacing the Windows Update drivers with Nvidia's drivers. Be warned though, it is not as straightforward as it should be.
See here:
https://faceswap.dev/forum/viewtopic.php?f=5&t=14
Re: Is it necessary detecting GPU&driver for use?
Posted: Thu Jul 18, 2019 8:10 pm
by HeyThereGoesGuy
I'm running into issues with it detecting my drivers as well. I also have nvidia-smi.exe in my sys32 folder and not under Program Files.
My drivers were pre-installed as the "DCH" version so I'm going to try and remove them and re-install the "Standar" drivers per your instructions on the other thread.
CUDA is working fine when training though. (I only have a 1050ti so my batch size has to be small. Still works though!)
Here is my sysinfo:
Code: Select all
============ System Information ============
encoding: cp1252
git_branch: master
git_commits: 5da91b7 Add .ico back for legacy windows installs. 09c3215 Update README.md. 5f73418 Update icons. 0a82e51 Bugflx: Correct input variable to res_block. ae259e9 Add .mpg file extension support
gpu_cuda: 10.0
gpu_cudnn: 7.5.0
gpu_devices:
gpu_devices_active:
gpu_driver: No Nvidia driver found
gpu_vram:
os_machine: AMD64
os_platform: Windows-10-10.0.18362-SP0
os_release: 10
py_command: C:\faceswap/faceswap.py gui
py_conda_version: conda 4.5.12
py_implementation: CPython
py_version: 3.6.8
py_virtual_env: True
sys_cores: 12
sys_processor: Intel64 Family 6 Model 158 Stepping 10, GenuineIntel
sys_ram: Total: 32496MB, Available: 17093MB, Used: 15403MB, Free: 17093MB
=============== Pip Packages ===============
absl-py==0.7.1
astor==0.7.1
certifi==2019.6.16
cloudpickle==1.1.1
cycler==0.10.0
cytoolz==0.9.0.1
dask==1.2.2
decorator==4.4.0
dlib==19.17.99
fastcluster==1.1.25
ffmpy==0.2.2
gast==0.2.2
grpcio==1.16.1
h5py==2.9.0
imageio==2.5.0
imageio-ffmpeg==0.3.0
joblib==0.13.2
Keras==2.2.4
Keras-Applications==1.0.8
Keras-Preprocessing==1.1.0
kiwisolver==1.1.0
Markdown==3.1.1
matplotlib==2.2.2
mkl-fft==1.0.12
mkl-random==1.0.2
mkl-service==2.0.2
mock==3.0.5
networkx==2.3
numpy==1.16.2
nvidia-ml-py3==7.352.0
olefile==0.46
opencv-python==4.1.0.25
pathlib==1.0.1
Pillow==6.0.0
protobuf==3.8.0
psutil==5.6.2
pynvx==1.0.0
pyparsing==2.4.0
pyreadline==2.1
python-dateutil==2.8.0
pytz==2019.1
PyWavelets==1.0.3
pywin32==223
PyYAML==5.1
scikit-image==0.15.0
scikit-learn==0.21.2
scipy==1.2.1
six==1.12.0
tensorboard==1.13.1
tensorflow==1.13.1
tensorflow-estimator==1.13.0
termcolor==1.1.0
toolz==0.9.0
toposort==1.5
tornado==6.0.2
tqdm==4.32.1
Werkzeug==0.15.4
wincertstore==0.2
============== Conda Packages ==============
# packages in environment at C:\Users\enegendank\MiniConda3\envs\fs-gpu:
#
# Name Version Build Channel
_tflow_select 2.1.0 gpu
absl-py 0.7.1 py36_0
astor 0.7.1 py36_0
blas 1.0 mkl
ca-certificates 2019.6.16 hecc5488_0 conda-forge
certifi 2019.6.16 py36_1 conda-forge
cloudpickle 1.1.1 py_0
cudatoolkit 10.0.130 0
cudnn 7.6.0 cuda10.0_0
cycler 0.10.0 py36h009560c_0
cytoolz 0.9.0.1 py36hfa6e2cd_1
dask-core 1.2.2 py_0
decorator 4.4.0 py36_1
dlib 19.17.99 <pip>
fastcluster 1.1.25 <pip>
ffmpy 0.2.2 <pip>
freetype 2.9.1 ha9979f8_1
gast 0.2.2 py36_0
grpcio 1.16.1 py36h351948d_1
h5py 2.9.0 py36h5e291fa_0
hdf5 1.10.4 h7ebc959_0
icc_rt 2019.0.0 h0cc432a_1
icu 58.2 ha66f8fd_1
imageio 2.5.0 py36_0
imageio-ffmpeg 0.3.0 <pip>
intel-openmp 2019.4 245
joblib 0.13.2 py36_0
jpeg 9c hfa6e2cd_1001 conda-forge
keras 2.2.4 0
keras-applications 1.0.8 py_0
keras-base 2.2.4 py36_0
keras-preprocessing 1.1.0 py_1
kiwisolver 1.1.0 py36ha925a31_0
libblas 3.8.0 8_mkl conda-forge
libcblas 3.8.0 8_mkl conda-forge
liblapack 3.8.0 8_mkl conda-forge
liblapacke 3.8.0 8_mkl conda-forge
libmklml 2019.0.3 0
libpng 1.6.37 h2a8f88b_0
libprotobuf 3.8.0 h7bd577a_0
libtiff 4.0.10 hb898794_2
libwebp 1.0.2 hfa6e2cd_2 conda-forge
markdown 3.1.1 py36_0
matplotlib 2.2.2 py36had4c4a9_2
mkl 2019.4 245
mkl-service 2.0.2 py36he774522_0
mkl_fft 1.0.12 py36h14836fe_0
mkl_random 1.0.2 py36h343c172_0
mock 3.0.5 py36_0
networkx 2.3 py_0
numpy 1.16.2 py36h19fb1c0_0
numpy-base 1.16.2 py36hc3f5095_0
nvidia-ml-py3 7.352.0 <pip>
olefile 0.46 py36_0
opencv 4.1.0 py36hb4945ee_5 conda-forge
opencv-python 4.1.0.25 <pip>
openssl 1.1.1c hfa6e2cd_0 conda-forge
pathlib 1.0.1 py36_1
pillow 6.0.0 py36hdc69c19_0
pip 19.1.1 py36_0
protobuf 3.8.0 py36h33f27b4_0
psutil 5.6.2 py36he774522_0
pynvx 1.0.0 <pip>
pyparsing 2.4.0 py_0
pyqt 5.9.2 py36h6538335_2
pyreadline 2.1 py36_1
python 3.6.8 h9f7ef89_7
python-dateutil 2.8.0 py36_0
pytz 2019.1 py_0
pywavelets 1.0.3 py36h8c2d366_1
pywin32 223 py36hfa6e2cd_1
pyyaml 5.1 py36he774522_0
qt 5.9.7 vc14h73c81de_0
scikit-image 0.15.0 py36ha925a31_0
scikit-learn 0.21.2 py36h6288b17_0
scipy 1.2.1 py36h29ff71c_0
setuptools 41.0.1 py36_0
sip 4.19.8 py36h6538335_0
six 1.12.0 py36_0
sqlite 3.28.0 he774522_0
tensorboard 1.13.1 py36h33f27b4_0
tensorflow 1.13.1 gpu_py36h9006a92_0
tensorflow-base 1.13.1 gpu_py36h871c8ca_0
tensorflow-estimator 1.13.0 py_0
tensorflow-gpu 1.13.1 h0d30ee6_0
termcolor 1.1.0 py36_1
tk 8.6.8 hfa6e2cd_0
toolz 0.9.0 py36_0
toposort 1.5 <pip>
tornado 6.0.2 py36he774522_0
tqdm 4.32.1 py_0
vc 14.1 h0510ff6_4
vs2015_runtime 14.15.26706 h3a45250_4
werkzeug 0.15.4 py_0
wheel 0.33.4 py36_0
wincertstore 0.2 py36h7fe50ca_0
xz 5.2.4 h2fa13f4_4
yaml 0.1.7 hc54c509_2
zlib 1.2.11 h62dcd97_3
zstd 1.3.7 h508b16e_0
Re: Is it necessary detecting GPU&driver for use?
Posted: Thu Jul 18, 2019 10:46 pm
by torzdf
We are looking into this. The problem is upstream of us with how the pynvml module (module for reading Nvidia driver information) gets its information. It expects it the information to be in the Nvidia location, not the Windows location.
If you can get the Standard drivers installed, that will definitely be the quickest solution, but we are trying to get a long term fix in for this.
Re: "NVML Shared Library Not Found" with new drivers.
Posted: Thu Jul 18, 2019 10:51 pm
by ChilledGumbo
Yep, seems to work for me!
Thanks
Re: Is it necessary detecting GPU&driver for use?
Posted: Thu Jul 18, 2019 11:00 pm
by HeyThereGoesGuy
The standard drivers worked. It still seems like mtcnn is using the CPU for training then Fan is using CUDA. Does that sound possible? I have high CPU usage searching for images. When it finds one, I get a small amount of GPU usage then back to idle.
Re: "NVML Shared Library Not Found" with new drivers.
Posted: Thu Jul 18, 2019 11:07 pm
by pfakanator
I've had success fixing this by doing the following:
Download current game ready driver (Not Experience!)
Download DDU and reboot in safe mode.
For Windows 10 safe mode follow these instructions.
https://support.microsoft.com/en-us/hel ... -safe-mode
Run DDU
In options, make sure all Nvidia Specific Options are selected.
Note: In the options for the program, under Advanced Options check "Prevent downloads of drivers from "Windows update" when "Windows" search for a driver for a device. You can re-enable this after everything is fixed.
Gut everything and perform "Clean and restart"
Log in normally and install your freshly downloaded Nvidia drivers.
Reboot and give FaceSwap a whirl.
To re-enable driver downloads from windows, run DDU again (this can be done w/out safe mode). Click the launch button and ignore the safe mode warning. Navigate, once again, to the options menu and uncheck "Prevent downloads of drivers from "Windows update" when "Windows" search for a driver for a device.
I hope this helps.
Re: Is it necessary detecting GPU&driver for use?
Posted: Thu Jul 18, 2019 11:26 pm
by bryanlyon
Detection takes longer, it's a harder problem iterating over the image in multiple resolutions to find faces. It's more than 10x longer to do than landmarks. That's all.
Re: Is it necessary detecting GPU&driver for use?
Posted: Thu Jul 18, 2019 11:55 pm
by chrisume
I don't know if you managed to get this fixed.
But I did cleanup my graphical drivers with DDU (display driver uninstaller)
you can find that @ https://www.guru3d.com/files-details/di ... nload.html
After that I downloaded geforce experience to get my latest driver installed.
This fixed my gpu detection problem.
Re: "NVML Shared Library Not Found" with new drivers.
Posted: Fri Jul 19, 2019 1:08 am
by torzdf
We have found a potential solution to this issue, but need to test it works before pushing to the main repo.
If you are having this issue, please try the following:
Open up an Anaconda Prompt:
Start > Anaconda Prompt
Then execute the following commands:
Code: Select all
conda activate faceswap
pip uninstall nvidia-ml-py3
pip install git+https://github.com/nicolargo/nvidia-ml-py3
Exit the Anaconda prompt and re-launch Faceswap and try again
Please confirm if this solution works or not, as we would like to add it to our codebase if it does.
Re: "NVML Shared Library Not Found" with new drivers.
Posted: Fri Jul 19, 2019 8:15 pm
by ChilledGumbo
Would there be any way to do this using the Installer version of faceswap for windows?
Re: "NVML Shared Library Not Found" with new drivers.
Posted: Fri Jul 19, 2019 8:16 pm
by bryanlyon
Yes, if we get confirmation that it works with users, we can implement it automatically. But we want to be sure it fixes the problem before we put in the workaround.
Re: "NVML Shared Library Not Found" with new drivers.
Posted: Fri Jul 19, 2019 8:19 pm
by ChilledGumbo
Ok, then. I'll install on windows the manual way and see if this works.
Re: "NVML Shared Library Not Found" with new drivers.
Posted: Fri Jul 19, 2019 8:27 pm
by torzdf
You don't need to install the manual way.
Installing with the installer will be fine to follow these instructions
Re: "NVML Shared Library Not Found" with new drivers.
Posted: Fri Jul 19, 2019 9:38 pm
by ChilledGumbo
Okay, yes, this new method seems to work fine.
Thank you.