GPU Not being Used

Installing and setting up FaceSwap


Forum rules

Read the FAQs and search the forum before posting a new topic.

Please mark any answers that fixed your problems so others can find the solutions.

Locked
User avatar
363LS2GTO
Posts: 30
Joined: Fri Jul 08, 2022 7:06 pm
Has thanked: 1 time
Been thanked: 4 times

GPU Not being Used

Post by 363LS2GTO »

Hello,

I installed face swap for my Windows 10 system and while I have it running, it is not utilizing the meager 1030 2GB GDDR5 NVIDIA GPU I have. It is running fine via the CPU but I don't want it constantly running near 100% I have verified the GPU is not working in CUDA view under task manager.

I have updated Windows, I have updated my NVIDIA graphics card to the most recent driver, and I have reinstalled face swap. I read the entire download and there appeared to be no problems.

Despite instructions on the forum, I have been unable to obtain any information from the GUI "Output system information" (nothing happens and I can't find an output anywhere) nor have I had luck with the command prompt command. I do have a crash log however.

The program states "setting backend to NVIDIA," The crash log states that "Tenserflow GPU - == 2.8.2" and under "Conda packages" cudnn and cudatoolkit are installed. I don't see any references to "conda cuda" by itself. Last, no global packages are listed for gpu : cuda or gpu: cudnn.

Any idea what could be wrong? Based upon other posts I have read, most everything seems to be in order. I am not sure about the lack of "condu cuda" under "conda packages."

Not sure why the processor is listed as an AMD as I have an Intel chip.

Thank you.

[media]

Code: Select all

============ System Information ============
encoding:            cp1252
git_branch:          master
git_commits:         9945efb Bugfix: Phaze-A Fix learn mask for upscales in FC
gpu_cuda:            No global version found. Check Conda packages for Conda Cuda
gpu_cudnn:           No global version found. Check Conda packages for Conda cuDNN
gpu_devices:         GPU_0: NVIDIA GeForce GT 1030
gpu_devices_active:  GPU_0
gpu_driver:          471.41
gpu_vram:            GPU_0: 2048MB
os_machine:          AMD64
os_platform:         Windows-10-10.0.19044-SP0
os_release:          10
py_command:          C:\Users\cpwan\faceswap\faceswap.py train -A E:/Fakes A/Faces -B E:/Fakes Megan B/faces -m E:/Model Megan -t original -bs 10 -it 1000000 -s 10 -ss 25000 -L INFO -LF E:/Fakes saved config for model/log file friday.log -gui
py_conda_version:    conda 4.13.0
py_implementation:   CPython
py_version:          3.9.12
py_virtual_env:      True
sys_cores:           2
sys_processor:       Intel64 Family 6 Model 94 Stepping 3, GenuineIntel
sys_ram:             Total: 8145MB, Available: 3337MB, Used: 4807MB, Free: 3337MB

=============== Pip Packages ===============
absl-py==1.1.0
astunparse==1.6.3
cachetools==5.2.0
certifi==2022.6.15
charset-normalizer==2.1.0
cloudpickle==2.1.0
colorama @ file:///C:/Windows/TEMP/abs_9439aeb1-0254-449a-96f7-33ab5eb17fc8apleb4yn/croots/recipe/colorama_1657009099097/work
cycler @ file:///tmp/build/80754af9/cycler_1637851556182/work
decorator==5.1.1
dm-tree==0.1.7
fastcluster @ file:///D:/bld/fastcluster_1649783471014/work
ffmpy==0.2.3
flatbuffers==2.0
fonttools==4.25.0
gast==0.5.3
google-auth==2.9.0
google-auth-oauthlib==0.4.6
google-pasta==0.2.0
grpcio==1.47.0
h5py==3.7.0
idna==3.3
imageio @ file:///tmp/build/80754af9/imageio_1617700267927/work
imageio-ffmpeg @ file:///home/conda/feedstock_root/build_artifacts/imageio-ffmpeg_1649960641006/work
importlib-metadata==4.12.0
joblib @ file:///tmp/build/80754af9/joblib_1635411271373/work
keras==2.8.0
Keras-Preprocessing==1.1.2
kiwisolver @ file:///C:/ci/kiwisolver_1653292407425/work
libclang==14.0.1
Markdown==3.3.7
matplotlib @ file:///C:/ci/matplotlib-suite_1647423638658/work
mkl-fft==1.3.1
mkl-random @ file:///C:/ci/mkl_random_1626186184308/work
mkl-service==2.4.0
munkres==1.1.4
numpy @ file:///C:/ci/numpy_and_numpy_base_1652784039997/work
nvidia-ml-py==11.510.69
oauthlib==3.2.0
opencv-python==4.6.0.66
opt-einsum==3.3.0
packaging @ file:///tmp/build/80754af9/packaging_1637314298585/work
Pillow==9.0.1
protobuf==3.19.4
psutil @ file:///C:/Windows/Temp/abs_b2c2fd7f-9fd5-4756-95ea-8aed74d0039flsd9qufz/croots/recipe/psutil_1656431277748/work
pyasn1==0.4.8
pyasn1-modules==0.2.8
pyparsing @ file:///tmp/build/80754af9/pyparsing_1635766073266/work
python-dateutil @ file:///tmp/build/80754af9/python-dateutil_1626374649649/work
pywin32==302
requests==2.28.1
requests-oauthlib==1.3.1
rsa==4.8
scikit-learn @ file:///C:/ci/scikit-learn_1642617276183/work
scipy @ file:///C:/ci/scipy_1641555170412/work
sip==4.19.13
six @ file:///tmp/build/80754af9/six_1644875935023/work
tensorboard==2.8.0
tensorboard-data-server==0.6.1
tensorboard-plugin-wit==1.8.1
tensorflow-estimator==2.8.0
tensorflow-gpu==2.8.2
tensorflow-io-gcs-filesystem==0.26.0
tensorflow-probability==0.16.0
termcolor==1.1.0
threadpoolctl @ file:///Users/ktietz/demo/mc3/conda-bld/threadpoolctl_1629802263681/work
tornado @ file:///C:/ci/tornado_1606924294691/work
tqdm @ file:///C:/ci/tqdm_1650636210717/work
typing_extensions @ file:///opt/conda/conda-bld/typing_extensions_1647553014482/work
urllib3==1.26.10
Werkzeug==2.1.2
wincertstore==0.2
wrapt==1.14.1
zipp==3.8.0

============== Conda Packages ==============
# packages in environment at C:\Users\cpwan\MiniConda3\envs\faceswap:
#
# Name                    Version                   Build  Channel
absl-py                   1.1.0                    pypi_0    pypi
astunparse                1.6.3                    pypi_0    pypi
blas                      1.0                         mkl  
brotli                    1.0.9                ha925a31_2  
ca-certificates           2022.6.15            h5b45459_0    conda-forge
cachetools                5.2.0                    pypi_0    pypi
certifi                   2022.6.15        py39hcbf5309_0    conda-forge
charset-normalizer        2.1.0                    pypi_0    pypi
cloudpickle               2.1.0                    pypi_0    pypi
colorama                  0.4.5            py39haa95532_0  
cudatoolkit               11.2.2              h933977f_10    conda-forge
cudnn                     8.1.0.77             h3e0f4f4_0    conda-forge
cycler                    0.11.0             pyhd3eb1b0_0  
decorator                 5.1.1                    pypi_0    pypi
dm-tree                   0.1.7                    pypi_0    pypi
fastcluster               1.2.6            py39h2e25243_1    conda-forge
ffmpeg                    4.3.1                ha925a31_0    conda-forge
ffmpy                     0.2.3                    pypi_0    pypi
flatbuffers               2.0                      pypi_0    pypi
fonttools                 4.25.0             pyhd3eb1b0_0  
freetype                  2.10.4               hd328e21_0  
gast                      0.5.3                    pypi_0    pypi
git                       2.34.1               haa95532_0  
google-auth               2.9.0                    pypi_0    pypi
google-auth-oauthlib      0.4.6                    pypi_0    pypi
google-pasta              0.2.0                    pypi_0    pypi
grpcio                    1.47.0                   pypi_0    pypi
h5py                      3.7.0                    pypi_0    pypi
icc_rt                    2019.0.0             h0cc432a_1  
icu                       58.2                 ha925a31_3  
idna                      3.3                      pypi_0    pypi
imageio                   2.9.0              pyhd3eb1b0_0  
imageio-ffmpeg            0.4.7              pyhd8ed1ab_0    conda-forge
importlib-metadata        4.12.0                   pypi_0    pypi
intel-openmp              2021.4.0          haa95532_3556  
joblib                    1.1.0              pyhd3eb1b0_0  
jpeg                      9e                   h2bbff1b_0  
keras                     2.8.0                    pypi_0    pypi
keras-preprocessing       1.1.2                    pypi_0    pypi
kiwisolver                1.4.2            py39hd77b12b_0  
libclang                  14.0.1                   pypi_0    pypi
libpng                    1.6.37               h2a8f88b_0  
libtiff                   4.2.0                he0120a3_1  
libwebp                   1.2.2                h2bbff1b_0  
lz4-c                     1.9.3                h2bbff1b_1  
markdown                  3.3.7                    pypi_0    pypi
matplotlib                3.5.1            py39haa95532_1  
matplotlib-base           3.5.1            py39hd77b12b_1  
mkl                       2021.4.0           haa95532_640  
mkl-service               2.4.0            py39h2bbff1b_0  
mkl_fft                   1.3.1            py39h277e83a_0  
mkl_random                1.2.2            py39hf11a4ad_0  
munkres                   1.1.4                      py_0  
numpy                     1.22.3           py39h7a0a035_0  
numpy-base                1.22.3           py39hca35cd5_0  
nvidia-ml-py              11.510.69                pypi_0    pypi
oauthlib                  3.2.0                    pypi_0    pypi
opencv-python             4.6.0.66                 pypi_0    pypi
openssl                   1.1.1q               h8ffe710_0    conda-forge
opt-einsum                3.3.0                    pypi_0    pypi
packaging                 21.3               pyhd3eb1b0_0  
pillow                    9.0.1            py39hdc2b20a_0  
pip                       21.2.4           py39haa95532_0  
protobuf                  3.19.4                   pypi_0    pypi
psutil                    5.9.0            py39h2bbff1b_0  
pyasn1                    0.4.8                    pypi_0    pypi
pyasn1-modules            0.2.8                    pypi_0    pypi
pyparsing                 3.0.4              pyhd3eb1b0_0  
pyqt                      5.9.2            py39hd77b12b_6  
python                    3.9.12               h6244533_0  
python-dateutil           2.8.2              pyhd3eb1b0_0  
python_abi                3.9                      2_cp39    conda-forge
pywin32                   302              py39h2bbff1b_2  
qt                        5.9.7            vc14h73c81de_0  
requests                  2.28.1                   pypi_0    pypi
requests-oauthlib         1.3.1                    pypi_0    pypi
rsa                       4.8                      pypi_0    pypi
scikit-learn              1.0.2            py39hf11a4ad_1  
scipy                     1.7.3            py39h0a974cb_0  
setuptools                61.2.0           py39haa95532_0  
sip                       4.19.13          py39hd77b12b_0  
six                       1.16.0             pyhd3eb1b0_1  
sqlite                    3.38.5               h2bbff1b_0  
tensorboard               2.8.0                    pypi_0    pypi
tensorboard-data-server   0.6.1                    pypi_0    pypi
tensorboard-plugin-wit    1.8.1                    pypi_0    pypi
tensorflow-estimator      2.8.0                    pypi_0    pypi
tensorflow-gpu            2.8.2                    pypi_0    pypi
tensorflow-io-gcs-filesystem 0.26.0                   pypi_0    pypi
tensorflow-probability    0.16.0                   pypi_0    pypi
termcolor                 1.1.0                    pypi_0    pypi
threadpoolctl             2.2.0              pyh0d69192_0  
tk                        8.6.12               h2bbff1b_0  
tornado                   6.1              py39h2bbff1b_0  
tqdm                      4.64.0           py39haa95532_0  
typing-extensions         4.1.1                hd3eb1b0_0  
typing_extensions         4.1.1              pyh06a4308_0  
tzdata                    2022a                hda174b7_0  
urllib3                   1.26.10                  pypi_0    pypi
vc                        14.2                 h21ff451_1  
vs2015_runtime            14.27.29016          h5e58377_2  
werkzeug                  2.1.2                    pypi_0    pypi
wheel                     0.37.1             pyhd3eb1b0_0  
wincertstore              0.2              py39haa95532_2  
wrapt                     1.14.1                   pypi_0    pypi
xz                        5.2.5                h8cc25b3_1  
zipp                      3.8.0                    pypi_0    pypi
zlib                      1.2.12               h8cc25b3_2  
zstd                      1.5.2                h19a0ad4_0  

=============== State File =================
User avatar
torzdf
Posts: 2649
Joined: Fri Jul 12, 2019 12:53 am
Answers: 159
Has thanked: 128 times
Been thanked: 622 times

Re: GPU Not being Used

Post by torzdf »

Cuda and cudnn are installed fine in your environment:

Code: Select all

cudatoolkit 11.2.2 h933977f_10 conda-forge
cudnn 8.1.0.77 h3e0f4f4_0 conda-forge

The first step is to always to do this, it solves 99% of issues, but only if you don't skip any steps:
app.php/faqpage#f1r1

However, with a 2GB GPU you are not going to be able to do very much, particularly under Windows, so I'm not sure how much effort it is worth going through to resolve your issues.

My word is final

User avatar
363LS2GTO
Posts: 30
Joined: Fri Jul 08, 2022 7:06 pm
Has thanked: 1 time
Been thanked: 4 times

Re: GPU Not being Used

Post by 363LS2GTO »

I will try the steps you listed again.

I may be starting with a 2gb GPU but it will be a lot faster than CPU. I have lots of time.

If I like the results, I certainly can afford to purchase a more powerful graphic card. If I never get the program working correctly, no point in that as I don't intend on destroying my CPU using the wrong tool for the job.

I will report back.

Thank you.

User avatar
torzdf
Posts: 2649
Joined: Fri Jul 12, 2019 12:53 am
Answers: 159
Has thanked: 128 times
Been thanked: 622 times

Re: GPU Not being Used

Post by torzdf »

It's not speed that will be the issue. It's unlikely to be enough VRAM (under Windows) to load any models. You might be able to run the lightweight model, but I would rank your chances at 'slim'.

Also, this:
https://www.quora.com/Does-Nvidia%E2%80 ... rogramming

My word is final

User avatar
363LS2GTO
Posts: 30
Joined: Fri Jul 08, 2022 7:06 pm
Has thanked: 1 time
Been thanked: 4 times

Re: GPU Not being Used

Post by 363LS2GTO »

Yes, just as explained in your FAQ I intend to use the lightweight model mostly as it states that 2GB GDDR5 is the minimum requirement. I thought I read that if the graphics ram is exceeded the program will offload work to the system ram / processor but was not clear if this would completely negate the graphics card.

If windows is going to be an issue, I will install it on linux. I have been using linux as my daily operating system for over ten years so that should not be a problem. I just had a fresh install of windows 10 ready to go and in the past graphics cards and linux did not always play nice together.

I went through your steps exactly, and same result. Is there a way to check and see if the graphics ram is overloaded? The current clip I am working on is six seconds long and the second model is based on about 30 photographs. Its a good match of simple frontal head views and is actually looking pretty good to me despite the "low' number of extract faces. My expectations are not high with my current card. I would just like to know the program works if I wanted to go further.

If I had been told 4gb of graphics ram was required other than CPU from the get go it would have saved me some frustration.

Thank you.

User avatar
torzdf
Posts: 2649
Joined: Fri Jul 12, 2019 12:53 am
Answers: 159
Has thanked: 128 times
Been thanked: 622 times

Re: GPU Not being Used

Post by torzdf »

It is possible to train lightweight on 2GB Vram, but I don' t like to guarantee it (it was built and tested on 1.6GB a long time ago). However, every system is different, and Tensorflow/Cuda will change requirements up and down on different iterations for the same task.

System RAM is never shared. You either train on GPU or you train on CPU. There is no sharing of RAM (for one thing it would be painfully slow shuffling that data backwards and forwards.)

Windows will always reserve more GPU VRAM than Linux. Most likely it will gobble up too much VRAM for you to be able to train a model, hence the Linux recommendation, however this is still no guarantee for the aforementioned reasons.

It would be possible to create a model using the Phaze-A plugin which uses even less VRAM, but that is far away from an entry-level model (in terms of complexity to configure).

363LS2GTO wrote: Sat Jul 09, 2022 1:35 am

I went through your steps exactly, and same result. Is there a way to check and see if the graphics ram is overloaded?

Yes. nvidia-smi is the way to see, accurately, how much VRAM is being used and what your GPU utilization is. However, you would know if you were overloaded, as the process would error out with an "out of memory" error. It would not simply ignore the GPU.

The current clip I am working on is six seconds long and the second model is based on about 30 photographs.

If this is all the training data you have, then this will be noway near enough to do a decent swap.

However, all this is moot. As per my earlier message, the GTX 1030 is not listed as a supported Cuda card by Nvidia. Most likely the reason that your card is not being used is that it is simply unsupported. More information (and workarounds) here:

This is an old article (from around the time of the card's release), so there is no reason to believe anything has changed in the meantime. However, the tl:dr conclusions drawn are:

Conclusion

You can’t do much more than watching movies with this board unfortunately. If you want to do some deep learning but have little money to spend, then even this is not the best option.

You can find the GTX 1050 Ti, which doubles all the specs for double of the money at $150. Not too expensive considering 2GB of RAM today seems to be really the bare minimum.

If can spend up to $300, then the latest GTX 1060 6GB with extended memory bandwidth will be your best option. It’s what I used for recent posts.

That being said, the GT 1030 carries some interest if:

  • You have a requirement on the passive cooling system
  • You have a single slot requirement in density environments

To be honest, it really wouldn't be worth the hassle to go through all those steps for a configuration that probably still won't work.

The best I could recommend is installing the "AMD" version of the software. Again. no guarantee that it will work, and you will have a more limited feature set. However. The AMD version is "GPU Agnostic", using OpenCL rather than Cuda. Given that Nvidia do not support Cuda on that card, it is the easiest way to test for a working GPU version.

Do note, however, that OpenCL is significantly slower than Cuda (albeit still significantly faster than CPU). If I were you, this is the route that I would take, for testing purposes, if nothing else.

Also note that model files trained on the AMD version are not compatible with the Nvidia version (i.e. you couldn't train the model under OpenCL and then continue it/convert with Cuda).

My word is final

User avatar
363LS2GTO
Posts: 30
Joined: Fri Jul 08, 2022 7:06 pm
Has thanked: 1 time
Been thanked: 4 times

Re: GPU Not being Used

Post by 363LS2GTO »

I appreciate the reply. I was under the impression that CUDA basically worked seamlessly with all modern NVIDIA cards / windows but as it seems that is not the case I will try the AMD method you suggested.

Graphics card prices seem to be dropping some and I have found several deals around $320-$340 for 8GB 3050s.

Thank you.

Locked