Faceswap Forum

Good to know you can make it work!
Sad to see it neuters your 2080ti performance in FS.
Just run the 3090 solo. It should be fast. Mine smokes for sure (almost literally)

hi guys please help .I followed gregormax guide, it worked I can extract both faces A and B but when I trained it i got an error immediately. thank you

Crash Report
https://justpaste.it/6jfl7

ryzen 9 3900x
rtx 3080
32gbram

Any idea when the new version that supports Ampere cards will be available? I've tried every guide in this thread to no avail. What is still needed for official support?

Nvidia and Tensorflow to actually support .
There are many of us that would like to see it fully supported.
Torzdf said if I give him 41 bitcoin, and give him 2 years to rewrite Tensorflow as a solo programmer .... I would be out a lot of money

Hi,
Just wanted to let you all know, that there is a possibility that the new faceswap update fixed some problems regarding the 30XX gpus. Before the update while training I had random crashes and for some reason it seemed like the core clock of my card was capped at around 600MHz. I updated today and now everything works. I have no idea why honestly, but just wanted to say that it runs at the expected iteration speed and core clock frequency.

Also if anyone's wandering I used Gregormax's tutorial and I have a i9-10900kf and a rtx 3090.

FWIW there has been no recent update that will likely have changed anything here. Most likely it's just because 30xx support in Tensorflow is currently "flaky".

Then I have literally no idea what happened, but I tried literally everything before and the result was the same. Only after the update it started working, I literally didn't change or download anything else. I just don't get how it always didn't work before and now it works flawlessly??

gregormax wrote: ↑Mon Mar 15, 2021 10:52 pm

Might help someone as it was a nightmare for me to make it work on an RTX3080 and finally I got it working. After many tries with conda, update'ing, upgrading, reinstalling, etc. here's what I've done and might help some (Windows 10):

1. Install the latest version of CUDA (in my case 11.2);
2. Install the latest version of Cudnn (in my case 8.1.1) - tip: don't forget there's an asterisk (*), so actually you need to copy all files from installation folders;
3. Restart;
4. Remove the faceswap folder (if you have installed it before);
5. Remove miniconda if you have installed it before;
6. Install faceswap and choose NVIDIA during installation (not CPU!);
7. Open the anaconda prompt window and enter commands (one by one and in case of brotli I just tried all options since some don't work);
   conda activate faceswap
   conda remove tensorflow
   conda install brotli
   conda install urllib3
   conda install -c anaconda urllib3
   pip install tensorflow-gpu==2.4.1
8. Run Faceswap and choose NVIDIA.

Works for me! In my case the difference is pretty huge. On CPU I was getting around 3-4 EGs/sec, now I'm getting much more (50-70 EGs/sec with 16-32 batch) - though I'm also using a bigger image file (1024) to train my model so I think this can have an influence on the speed of the process since it takes more memory. Anyway I can confirm it's possible to make it work on the RTX3080 on Win10.

Many thanks for this. Any idea why I'm getting this crash report? https://ufile.io/ij8cmrkt

I'm also getting a message CUBLAS_STATUS_NOT_INITIALIZED when I extract; however, the extraction will complete.

I was actually able to get this to work on my 3070 card using the instructions I quoted above. I kept getting various errors after I would install and then re-install following the same instructions. On like the 5th attempt, everything worked perfectly.

I have an option to buy a pc with rtx3090 24gb. I am trying to follow this topic but I am not sure, is all problems solved now for rtx30xx cards with the new version of FS? I dont want to dive deep inte cuda etc. its not my thing.

I am following the protocol for installing FS with a 30xx-card, how do I install cuDNN 8.0? I have downloaded the file and extracted but I dont see any .exe file?

rolnor wrote: ↑Wed Jul 07, 2021 3:18 pm

I am following the protocol for installing FS with a 30xx-card, how do I install cuDNN 8.0? I have downloaded the file and extracted but I dont see any .exe file?

Follow the instructions here: https://docs.nvidia.com/deeplearning/cu ... ll-windows

Thanx!
It works on a rtx3060 12gb card but is very slow, 8 times slower than a gtx1080 8gb card. I am running Dfaker 256pxs batchsize 3. If I use batchsize 60 its very, very slow. Can this be fixed? It does seem to be running on gpu, not cpu but I am not sure.

Many messages have been posted with re. getting the 30XX cards working on Linux.

We've got a fresh install of Ubuntu 20.04 but are having trouble getting our card working.

We followed the usual procedure, installed the latest CUDA (11.4)/CUDnn libraries/runtimes (8.2.2). We installed FaceSwap using the shell script installer and chose CPU.

We then removed tensorflow and installed tensorflow-gpu 2.4.2.

We finally launches faceswap and chose NVIDIA and it recognized the GPU. However, the CUDA/CUDnn libraries are not recognized.

For some reason faceswap isn't recognizing our global env variables.

For example:

It thinks we're still on CUDA 11.0 and no global CUDnn libraries are noticed.

Are there some environment variables/globals we are missing?

nvidia-smi:

Excuse the ignorance, very new to all of this.

Thank you in advance for any help, pointers, advice and commands

A few things.

1) The reporting from Faceswap on global Cuda/cuDNN may not always be 100% accurate. We hook into known locations to check, but these can change

2) I don't believe you have used the correct versions for Cuda/cuDNN for the TF 2.4.2 pip package, but you'd need to double check. The versions should be identical as those that TF was compiled with

3) Environment variables should work fine

4) The Cuda version shown in Nvidia-SMI is not the version of Cuda installed, but the maximum version of Cuda supported by your driver.

Hmm, cant seem to get this to work with 3060 Ti and W10.

Tried multiple configurations. From TF tried with 2,4, 2.4.1, 2.4.2. From Cuda and Cudnn versions I tried 11.0/8.0 and 11.2/8.2 combinations. Also from faceswap installations tried with gpu and cpu installation (and the method from first post, cpu to nvidia).

Weird thing is extractions works and it seems to utilize cuda cores (you can check cuda load of the card from task manager). It does give CUBLAS_STATUS_NOT_INITIALIZED in beginning and end, but it does still work.

Training and conversion on the other hand always gives the same errors.

CRITICAL Error caught! Exiting...
Caught exception in thread: '_training_0'

tensorflow.python.framework.errors_impl.NotFoundError: 2 root error(s) found.
  (0) Not found:  No algorithm worked!
	 [[node original/encoder/conv_128_0_conv2d/Conv2D_1 (defined at \faceswap\plugins\train\trainer\_base.py:193) ]]
	 [[cond_2/then/_20/batch_decoder_b_loss/ReadVariableOp/_66]]
  (1) Not found:  No algorithm worked!
	 [[node original/encoder/conv_128_0_conv2d/Conv2D_1 (defined at \faceswap\plugins\train\trainer\_base.py:193) ]]

You can see the gpu vram and cuda load increase for few seconds until this errors out. Tried multiple training models (original, dfaker, lightweight...), same error.

Any suggestions?

torzdf wrote: ↑Sat Jul 10, 2021 10:22 am

A few things.

1) The reporting from Faceswap on global Cuda/cuDNN may not always be 100% accurate. We hook into known locations to check, but these can change

2) I don't believe you have used the correct versions for Cuda/cuDNN for the TF 2.4.2 pip package, but you'd need to double check. The versions should be identical as those that TF was compiled with

3) Environment variables should work fine

4) The Cuda version shown in Nvidia-SMI is not the version of Cuda installed, but the maximum version of Cuda supported by your driver.

quick question.

my face swap env automatically install in miniconda folder not that anaconda.

should I use miniconda prompt? or copy env to anaconda folder?

babala wrote: ↑Sun Jul 11, 2021 10:26 pm

torzdf wrote: ↑Sat Jul 10, 2021 10:22 am

A few things.

1) The reporting from Faceswap on global Cuda/cuDNN may not always be 100% accurate. We hook into known locations to check, but these can change

2) I don't believe you have used the correct versions for Cuda/cuDNN for the TF 2.4.2 pip package, but you'd need to double check. The versions should be identical as those that TF was compiled with

3) Environment variables should work fine

4) The Cuda version shown in Nvidia-SMI is not the version of Cuda installed, but the maximum version of Cuda supported by your driver.

quick question.

my face swap env automatically install in miniconda folder not that anaconda.

should I use miniconda prompt? or copy env to anaconda folder?

I don't think it matters. Anaconda is just Miniconda with more software included.

joezi wrote: ↑Sat Jul 10, 2021 8:00 pm

Hmm, cant seem to get this to work with 3060 Ti and W10.

Tried multiple configurations. From TF tried with 2,4, 2.4.1, 2.4.2. From Cuda and Cudnn versions I tried 11.0/8.0 and 11.2/8.2 combinations. Also from faceswap installations tried with gpu and cpu installation (and the method from first post, cpu to nvidia).

Weird thing is extractions works and it seems to utilize cuda cores (you can check cuda load of the card from task manager). It does give CUBLAS_STATUS_NOT_INITIALIZED in beginning and end, but it does still work.

Training and conversion on the other hand always gives the same errors.

Code: Select all

CRITICAL Error caught! Exiting...
Caught exception in thread: '_training_0'

Code: Select all

tensorflow.python.framework.errors_impl.NotFoundError: 2 root error(s) found.
  (0) Not found:  No algorithm worked!
	 [[node original/encoder/conv_128_0_conv2d/Conv2D_1 (defined at \faceswap\plugins\train\trainer\_base.py:193) ]]
	 [[cond_2/then/_20/batch_decoder_b_loss/ReadVariableOp/_66]]
  (1) Not found:  No algorithm worked!
	 [[node original/encoder/conv_128_0_conv2d/Conv2D_1 (defined at \faceswap\plugins\train\trainer\_base.py:193) ]]

You can see the gpu vram and cuda load increase for few seconds until this errors out. Tried multiple training models (original, dfaker, lightweight...), same error.

Any suggestions?

Enable "Allow-Growth" in Global Model Settings.

torzdf wrote: ↑Mon Jul 12, 2021 12:17 am

Enable "Allow-Growth" in Global Model Settings.

Thanks, but no change. Same errors.

Any news on the official support? It seems conda has added tensorflow 2.5 to their package repo week a go. Updating faceswap to support 2.5 would allow 11.2 cuda/8.2 cudnn combo to work.

edit: manaaged to get it to run once by having some background load on the card when starting training...

edit2: yeah, related to vram. I can get it to run by having something running in the background and then stopping it in the right point when the training process is running so that it wont go OOM.