[Guide] Using Faceswap on Nvidia RTX 30xx cards

Installing and setting up FaceSwap


Forum rules

Read the FAQs and search the forum before posting a new topic.

Please mark any answers that fixed your problems so others can find the solutions.

User avatar
abigflea
Posts: 176
Joined: Sat Feb 22, 2020 10:59 pm
Answers: 2
Has thanked: 20 times
Been thanked: 53 times

Re: [Guide] Using Faceswap on Nvidia RTX 30xx cards

Post by abigflea »

Good to know you can make it work!
Sad to see it neuters your 2080ti performance in FS.
Just run the 3090 solo. It should be fast. Mine smokes for sure (almost literally)

:o I dunno what I'm doing :shock:
RTX 3090 : RTX 3070 : RTX 3060 : RTX: 2070 : Ghetto 1060


Tags:

User avatar
mymooo3
Posts: 3
Joined: Tue Nov 12, 2019 3:51 am

Re: [Guide] Using Faceswap on Nvidia RTX 30xx cards

Post by mymooo3 »

hi guys please help .I followed gregormax guide, it worked I can extract both faces A and B but when I trained it i got an error immediately. thank you

Crash Report
https://justpaste.it/6jfl7

ryzen 9 3900x
rtx 3080
32gbram

Emergency food


User avatar
Megadead
Posts: 2
Joined: Mon Sep 16, 2019 7:27 pm

Re: [Guide] Using Faceswap on Nvidia RTX 30xx cards

Post by Megadead »

Any idea when the new version that supports Ampere cards will be available? I've tried every guide in this thread to no avail. What is still needed for official support?


User avatar
abigflea
Posts: 176
Joined: Sat Feb 22, 2020 10:59 pm
Answers: 2
Has thanked: 20 times
Been thanked: 53 times

Re: [Guide] Using Faceswap on Nvidia RTX 30xx cards

Post by abigflea »

Nvidia and Tensorflow to actually support .
There are many of us that would like to see it fully supported.
Torzdf said if I give him 41 bitcoin, and give him 2 years to rewrite Tensorflow as a solo programmer .... I would be out a lot of money

:o I dunno what I'm doing :shock:
RTX 3090 : RTX 3070 : RTX 3060 : RTX: 2070 : Ghetto 1060


User avatar
bolmin70
Posts: 2
Joined: Wed Apr 28, 2021 9:49 am

Re: [Guide] Using Faceswap on Nvidia RTX 30xx cards

Post by bolmin70 »

Hi,
Just wanted to let you all know, that there is a possibility that the new faceswap update fixed some problems regarding the 30XX gpus. Before the update while training I had random crashes and for some reason it seemed like the core clock of my card was capped at around 600MHz. I updated today and now everything works. I have no idea why honestly, but just wanted to say that it runs at the expected iteration speed and core clock frequency.

Also if anyone's wandering I used Gregormax's tutorial and I have a i9-10900kf and a rtx 3090.

Last edited by bolmin70 on Wed Apr 28, 2021 10:03 am, edited 1 time in total.

User avatar
torzdf
Posts: 1503
Joined: Fri Jul 12, 2019 12:53 am
Answers: 126
Has thanked: 54 times
Been thanked: 288 times

Re: [Guide] Using Faceswap on Nvidia RTX 30xx cards

Post by torzdf »

FWIW there has been no recent update that will likely have changed anything here. Most likely it's just because 30xx support in Tensorflow is currently "flaky".

My word is final


User avatar
bolmin70
Posts: 2
Joined: Wed Apr 28, 2021 9:49 am

Re: [Guide] Using Faceswap on Nvidia RTX 30xx cards

Post by bolmin70 »

Then I have literally no idea what happened, but I tried literally everything before and the result was the same. Only after the update it started working, I literally didn't change or download anything else. I just don't get how it always didn't work before and now it works flawlessly??


User avatar
ranmyaku
Posts: 2
Joined: Wed May 05, 2021 5:26 am

Re: [Guide] Using Faceswap on Nvidia RTX 30xx cards

Post by ranmyaku »

gregormax wrote: Mon Mar 15, 2021 10:52 pm

Might help someone as it was a nightmare for me to make it work on an RTX3080 and finally I got it working. After many tries with conda, update'ing, upgrading, reinstalling, etc. here's what I've done and might help some (Windows 10):

1. Install the latest version of CUDA (in my case 11.2);
2. Install the latest version of Cudnn (in my case 8.1.1) - tip: don't forget there's an asterisk (*), so actually you need to copy all files from installation folders;
3. Restart;
4. Remove the faceswap folder (if you have installed it before);
5. Remove miniconda if you have installed it before;
6. Install faceswap and choose NVIDIA during installation (not CPU!);
7. Open the anaconda prompt window and enter commands (one by one and in case of brotli I just tried all options since some don't work);
conda activate faceswap
conda remove tensorflow
conda install brotli
conda install urllib3
conda install -c anaconda urllib3
pip install tensorflow-gpu==2.4.1
8. Run Faceswap and choose NVIDIA.

Works for me! In my case the difference is pretty huge. On CPU I was getting around 3-4 EGs/sec, now I'm getting much more (50-70 EGs/sec with 16-32 batch) - though I'm also using a bigger image file (1024) to train my model so I think this can have an influence on the speed of the process since it takes more memory. Anyway I can confirm it's possible to make it work on the RTX3080 on Win10.

Many thanks for this. Any idea why I'm getting this crash report? https://ufile.io/ij8cmrkt

I'm also getting a message CUBLAS_STATUS_NOT_INITIALIZED when I extract; however, the extraction will complete.


User avatar
ranmyaku
Posts: 2
Joined: Wed May 05, 2021 5:26 am

Re: [Guide] Using Faceswap on Nvidia RTX 30xx cards

Post by ranmyaku »

I was actually able to get this to work on my 3070 card using the instructions I quoted above. I kept getting various errors after I would install and then re-install following the same instructions. On like the 5th attempt, everything worked perfectly.


User avatar
rolnor
Posts: 62
Joined: Tue Feb 23, 2021 10:09 am
Been thanked: 3 times

Re: [Guide] Using Faceswap on Nvidia RTX 30xx cards

Post by rolnor »

I have an option to buy a pc with rtx3090 24gb. I am trying to follow this topic but I am not sure, is all problems solved now for rtx30xx cards with the new version of FS? I dont want to dive deep inte cuda etc. its not my thing.


User avatar
rolnor
Posts: 62
Joined: Tue Feb 23, 2021 10:09 am
Been thanked: 3 times

CuDNN instal?

Post by rolnor »

I am following the protocol for installing FS with a 30xx-card, how do I install cuDNN 8.0? I have downloaded the file and extracted but I dont see any .exe file?


User avatar
bryanlyon
Site Admin
Posts: 634
Joined: Fri Jul 12, 2019 12:49 am
Answers: 41
Location: San Francisco
Has thanked: 3 times
Been thanked: 161 times
Contact:

Re: CuDNN instal?

Post by bryanlyon »

rolnor wrote: Wed Jul 07, 2021 3:18 pm

I am following the protocol for installing FS with a 30xx-card, how do I install cuDNN 8.0? I have downloaded the file and extracted but I dont see any .exe file?

Follow the instructions here: https://docs.nvidia.com/deeplearning/cu ... ll-windows


User avatar
rolnor
Posts: 62
Joined: Tue Feb 23, 2021 10:09 am
Been thanked: 3 times

Re: [Guide] Using Faceswap on Nvidia RTX 30xx cards

Post by rolnor »

Thanx!
It works on a rtx3060 12gb card but is very slow, 8 times slower than a gtx1080 8gb card. I am running Dfaker 256pxs batchsize 3. If I use batchsize 60 its very, very slow. Can this be fixed? It does seem to be running on gpu, not cpu but I am not sure.


User avatar
ianstephens
Posts: 67
Joined: Sun Feb 14, 2021 7:20 pm
Has thanked: 9 times

Getting 30XX Working on Linux

Post by ianstephens »

Many messages have been posted with re. getting the 30XX cards working on Linux.

We've got a fresh install of Ubuntu 20.04 but are having trouble getting our card working.

We followed the usual procedure, installed the latest CUDA (11.4)/CUDnn libraries/runtimes (8.2.2). We installed FaceSwap using the shell script installer and chose CPU.

We then removed tensorflow and installed tensorflow-gpu 2.4.2.

We finally launches faceswap and chose NVIDIA and it recognized the GPU. However, the CUDA/CUDnn libraries are not recognized.

For some reason faceswap isn't recognizing our global env variables.

For example:

example.jpg
example.jpg (40.71 KiB) Viewed 10009 times

It thinks we're still on CUDA 11.0 and no global CUDnn libraries are noticed.

Are there some environment variables/globals we are missing?

nvidia-smi:

smi.jpg
smi.jpg (62.66 KiB) Viewed 10008 times

Excuse the ignorance, very new to all of this.

Thank you in advance for any help, pointers, advice and commands :D


User avatar
torzdf
Posts: 1503
Joined: Fri Jul 12, 2019 12:53 am
Answers: 126
Has thanked: 54 times
Been thanked: 288 times

Re: [Guide] Using Faceswap on Nvidia RTX 30xx cards

Post by torzdf »

A few things.

1) The reporting from Faceswap on global Cuda/cuDNN may not always be 100% accurate. We hook into known locations to check, but these can change

2) I don't believe you have used the correct versions for Cuda/cuDNN for the TF 2.4.2 pip package, but you'd need to double check. The versions should be identical as those that TF was compiled with

3) Environment variables should work fine

4) The Cuda version shown in Nvidia-SMI is not the version of Cuda installed, but the maximum version of Cuda supported by your driver.

My word is final


User avatar
joezi
Posts: 4
Joined: Sat Jul 10, 2021 7:44 pm

Re: [Guide] Using Faceswap on Nvidia RTX 30xx cards

Post by joezi »

Hmm, cant seem to get this to work with 3060 Ti and W10.

Tried multiple configurations. From TF tried with 2,4, 2.4.1, 2.4.2. From Cuda and Cudnn versions I tried 11.0/8.0 and 11.2/8.2 combinations. Also from faceswap installations tried with gpu and cpu installation (and the method from first post, cpu to nvidia).

Weird thing is extractions works and it seems to utilize cuda cores (you can check cuda load of the card from task manager). It does give CUBLAS_STATUS_NOT_INITIALIZED in beginning and end, but it does still work.

Training and conversion on the other hand always gives the same errors.

Code: Select all

CRITICAL Error caught! Exiting...
Caught exception in thread: '_training_0'

Code: Select all

tensorflow.python.framework.errors_impl.NotFoundError: 2 root error(s) found.
  (0) Not found:  No algorithm worked!
	 [[node original/encoder/conv_128_0_conv2d/Conv2D_1 (defined at \faceswap\plugins\train\trainer\_base.py:193) ]]
	 [[cond_2/then/_20/batch_decoder_b_loss/ReadVariableOp/_66]]
  (1) Not found:  No algorithm worked!
	 [[node original/encoder/conv_128_0_conv2d/Conv2D_1 (defined at \faceswap\plugins\train\trainer\_base.py:193) ]]

You can see the gpu vram and cuda load increase for few seconds until this errors out. Tried multiple training models (original, dfaker, lightweight...), same error.

Any suggestions?


User avatar
babala
Posts: 8
Joined: Sun Jul 11, 2021 10:16 pm
Has thanked: 3 times

Re: [Guide] Using Faceswap on Nvidia RTX 30xx cards

Post by babala »

torzdf wrote: Sat Jul 10, 2021 10:22 am

A few things.

1) The reporting from Faceswap on global Cuda/cuDNN may not always be 100% accurate. We hook into known locations to check, but these can change

2) I don't believe you have used the correct versions for Cuda/cuDNN for the TF 2.4.2 pip package, but you'd need to double check. The versions should be identical as those that TF was compiled with

3) Environment variables should work fine

4) The Cuda version shown in Nvidia-SMI is not the version of Cuda installed, but the maximum version of Cuda supported by your driver.

quick question.

my face swap env automatically install in miniconda folder not that anaconda.

should I use miniconda prompt? or copy env to anaconda folder?


User avatar
torzdf
Posts: 1503
Joined: Fri Jul 12, 2019 12:53 am
Answers: 126
Has thanked: 54 times
Been thanked: 288 times

Re: [Guide] Using Faceswap on Nvidia RTX 30xx cards

Post by torzdf »

babala wrote: Sun Jul 11, 2021 10:26 pm
torzdf wrote: Sat Jul 10, 2021 10:22 am

A few things.

1) The reporting from Faceswap on global Cuda/cuDNN may not always be 100% accurate. We hook into known locations to check, but these can change

2) I don't believe you have used the correct versions for Cuda/cuDNN for the TF 2.4.2 pip package, but you'd need to double check. The versions should be identical as those that TF was compiled with

3) Environment variables should work fine

4) The Cuda version shown in Nvidia-SMI is not the version of Cuda installed, but the maximum version of Cuda supported by your driver.

quick question.

my face swap env automatically install in miniconda folder not that anaconda.

should I use miniconda prompt? or copy env to anaconda folder?

I don't think it matters. Anaconda is just Miniconda with more software included.

My word is final


User avatar
torzdf
Posts: 1503
Joined: Fri Jul 12, 2019 12:53 am
Answers: 126
Has thanked: 54 times
Been thanked: 288 times

Re: [Guide] Using Faceswap on Nvidia RTX 30xx cards

Post by torzdf »

joezi wrote: Sat Jul 10, 2021 8:00 pm

Hmm, cant seem to get this to work with 3060 Ti and W10.

Tried multiple configurations. From TF tried with 2,4, 2.4.1, 2.4.2. From Cuda and Cudnn versions I tried 11.0/8.0 and 11.2/8.2 combinations. Also from faceswap installations tried with gpu and cpu installation (and the method from first post, cpu to nvidia).

Weird thing is extractions works and it seems to utilize cuda cores (you can check cuda load of the card from task manager). It does give CUBLAS_STATUS_NOT_INITIALIZED in beginning and end, but it does still work.

Training and conversion on the other hand always gives the same errors.

Code: Select all

CRITICAL Error caught! Exiting...
Caught exception in thread: '_training_0'

Code: Select all

tensorflow.python.framework.errors_impl.NotFoundError: 2 root error(s) found.
  (0) Not found:  No algorithm worked!
	 [[node original/encoder/conv_128_0_conv2d/Conv2D_1 (defined at \faceswap\plugins\train\trainer\_base.py:193) ]]
	 [[cond_2/then/_20/batch_decoder_b_loss/ReadVariableOp/_66]]
  (1) Not found:  No algorithm worked!
	 [[node original/encoder/conv_128_0_conv2d/Conv2D_1 (defined at \faceswap\plugins\train\trainer\_base.py:193) ]]

You can see the gpu vram and cuda load increase for few seconds until this errors out. Tried multiple training models (original, dfaker, lightweight...), same error.

Any suggestions?

Enable "Allow-Growth" in Global Model Settings.

My word is final


User avatar
joezi
Posts: 4
Joined: Sat Jul 10, 2021 7:44 pm

Re: [Guide] Using Faceswap on Nvidia RTX 30xx cards

Post by joezi »

torzdf wrote: Mon Jul 12, 2021 12:17 am

Enable "Allow-Growth" in Global Model Settings.

Thanks, but no change. Same errors.

Any news on the official support? It seems conda has added tensorflow 2.5 to their package repo week a go. Updating faceswap to support 2.5 would allow 11.2 cuda/8.2 cudnn combo to work.

edit: manaaged to get it to run once by having some background load on the card when starting training...

edit2: yeah, related to vram. I can get it to run by having something running in the background and then stopping it in the right point when the training process is running so that it wont go OOM.

Last edited by joezi on Mon Jul 12, 2021 8:56 am, edited 1 time in total.

Post Reply