Hardware best practices

Talk about Hardware used for Deep Learning


User avatar
bryanlyon
Site Admin
Posts: 793
Joined: Fri Jul 12, 2019 12:49 am
Answers: 44
Location: San Francisco
Has thanked: 4 times
Been thanked: 218 times
Contact:

Hardware best practices

Post by bryanlyon »

Edit: At this time there are still issues with availability for GPUs. My "recommended" GPU is the 3060. WIth it's high vram and low MSRP it's the best value for new GPUs.

Buying hardware is always complicated, but it's even more complicated when your buying for faceswap. Because of that, I've put together this little guide for hardware.

First of, the most important thing with faceswap is your graphics card. This is by far the biggest effect in your swap speeds and results. (Note that quality is limited by your card but is extremely dependent on your data.)

For faceswap, there are 2 main things to consider on your gpu. 1: Nvidia is far superior to AMD (You may debate this statement when it comes to games, but Nvidia positively trounces on AMD in machine learning). 2: Your video card is incredibly important.

For these 2 reasons the card we recommend are as follows.

RTX 2060 Super.

This card's 8gb of ram and bargain price makes it the first stop for beginning ML tasks.

The speed increases from a 2070 or a 2080 isn't really much, usually on the order of 10-15%. If that's worth the 40-100% increase in cost, that's a decision for you to make.

RTX 2080 ti

The 11gb if vram on this card costs as much as 3 2060 supers, but if you want a single card option, it's the best way to beat its little brother.

RTX Titan

This card is stupid, you should only do this if you have more money than sense. It'd make more sense to buy 4 2060 supers and the 2 pcs to run them that this card would cost you. Speed is comparable to a 2080 ti, but at an enormous price premium. Unless you have other needs for this card DO NOT BUY IT (Full disclosure, I own this card).

RTX 2070

If you're willing to go used, check this card out for sure. But don't pay much more for it than you would for a 2060 super.

GTX 1070/1080/1080 ti

With RTX cards now at lower prices i wouldn't recommend these cards for a new build. Tensor cores have huge potential and these cards are just not as powerful for the price.

Your cpu matters far less, but still quite a bit for machine learning. We have all levels of cpus.

AMD ryzen 3xxx series.

Faceswap is multithreaded where it matters and so benefits from extra cores more than boost. For this reason i can't recommend any CPUs except for AMD's right now. Their 3700 and 3900 take the crown for best faceswap cpus out there for the price. You don't need a super powered 64 core monster, but feel free to look in that direction if that is what matters for you.

AMD threadripper

The only exception to this is if you want to maximize your pcie lanes and budget isn't a concern (make sure you donate) in which case a threadripper is your go-to. Nvidia doesn't use pcie 4.0 yet so you can get more bandwidth by using threadripper to provide all 16 Lanes to 3 cards or run 4 cards with at least 8 lanes each (some lanes are reserved for chipset and functionality).

Storage is not critical to faceswap, but it's best to keep it local. For this reason after a (preferably nvme) ssd boot drive, some good ole' spinning rust is fine for your data. Just try to keep other accesses to a minimum while training, so no watching 4k videos off the disk while you're training (a good idea to avoid any heavy videos while training since they'll hit the GPU too.

The rest of your gear should just focus on working with the hardware above, make sure you have enough pcie and memory slots on your motherboard. Check out pcpartpicker https://pcpartpicker.com for finding compatible hardware.

Don't go overboard though buying hardware you won't use like a $800 overclocking board. Keep it to what you'll actually use (in faceswap or other software)

Overclocking is bad for compute tasks like faceswap.

Which reminds me: do not overclock your GPU while training, in fact you should remove any factory overclocking and run it at Nvidia defaults to avoid model failures.

If you have any other hardware questions, please ask in the forum.

User avatar
madGANs
Posts: 4
Joined: Sat Aug 24, 2019 1:08 am
Has thanked: 1 time

Re: Hardware best practices

Post by madGANs »

Thanks for the share and the time to research/put it all down on here for us. This community is great.

User avatar
Linhaohoward
Posts: 23
Joined: Sat Dec 21, 2019 1:23 pm
Has thanked: 3 times

Re: Hardware best practices

Post by Linhaohoward »

bryanlyon wrote: Mon Jul 15, 2019 2:48 am

This card is stupid, you should only do this if you have more money than sense. It'd make more sense to buy 4 2060 supers and the 2 pcs to run them that this card would cost you. Speed is comparable to a 2080 ti, but at an enormous price premium. Unless you have other needs for this card DO NOT BUY IT (Full disclosure, I own this card).

Hi, not sure if this is a stupid question. But RTX 2060 isn’t SLI supported. How will this affect its performance?

User avatar
bryanlyon
Site Admin
Posts: 793
Joined: Fri Jul 12, 2019 12:49 am
Answers: 44
Location: San Francisco
Has thanked: 4 times
Been thanked: 218 times
Contact:

Re: Hardware best practices

Post by bryanlyon »

20xx series cards don't use SLI, but instead use nvlink. This is actually better, but it's not critical. Communication between the cards can happen through the pci-e bus or nvlink but sli won't work for non-gaming tasks.

However, consumer nvlink will only work between 2 cards anyway. So having 4 cards means you can't use nvlink. This was mainly a joke to show that an RTX Titan isn't a good purchase choice for Faceswap.

User avatar
Linhaohoward
Posts: 23
Joined: Sat Dec 21, 2019 1:23 pm
Has thanked: 3 times

Re: Hardware best practices

Post by Linhaohoward »

bryanlyon wrote: Sat Dec 21, 2019 5:07 pm

20xx series cards don't use SLI, but instead use nvlink. This is actually better, but it's not critical. Communication between the cards can happen through the pci-e bus or nvlink but sli won't work for non-gaming tasks.

However, consumer nvlink will only work between 2 cards anyway. So having 4 cards means you can't use nvlink. This was mainly a joke to show that an RTX Titan isn't a good purchase choice for Faceswap.

Hi Bryan, I’m actually making up my mind on the hardware build I’m purchasing. Sorry I should have been more specific. It’s rtx2070s, on Nvidia website under NVLink (SLI-ready) it says: Yes with rtx NVLink bridge.

Whereas the 2060s is a “No”

So I’m very confused.

Anyway you think I can train SAEHD 256 with these graphic cards? Perhaps at batchsize 4 or higher?

Also will like to hear your advise on other specs of the computer. An AI deeplearning techie told me that Cpu isn’t really important, an i5 and an i9 might make less than 10% difference in deep learning. And as for Ram he always just get the same amount as his gpu. Meaning to say I just need 16gb if I’m getting 2x rtx 2070s. But I do know different deeplearning have different needs of specs.

So what do you think is optimal for Deepfaking? If I’m not mistaken, Optimizer 3 uses cpu and ram on top of just gpu isn’t it? So wouldn’t cpu and ram play a more important role in Deepfake?

User avatar
bryanlyon
Site Admin
Posts: 793
Joined: Fri Jul 12, 2019 12:49 am
Answers: 44
Location: San Francisco
Has thanked: 4 times
Been thanked: 218 times
Contact:

Re: Hardware best practices

Post by bryanlyon »

It's important to note that DFL does not support multiple gpus, only Faceswap can use more than one GPU.

Cpu is still heavily used for things like augmentation, but it's far less important than the gpu.

User avatar
Linhaohoward
Posts: 23
Joined: Sat Dec 21, 2019 1:23 pm
Has thanked: 3 times

Re: Hardware best practices

Post by Linhaohoward »

bryanlyon wrote: Sun Dec 22, 2019 9:24 pm

It's important to note that DFL does not support multiple gpus, only Faceswap can use more than one GPU.

Cpu is still heavily used for things like augmentation, but it's far less important than the gpu.

So do you recommend 2x rtx2070s over a single 2080ti in Faceswap?

Can you advise if Ryzen 5 would be a sweet spot for cpu?

Btw how much Ram would I need? Does it matter in Optimizer mode 3 or does it have little to no effect?

User avatar
torzdf
Posts: 2651
Joined: Fri Jul 12, 2019 12:53 am
Answers: 159
Has thanked: 129 times
Been thanked: 622 times

Re: Hardware best practices

Post by torzdf »

This is the forum for faceswap, not DFL.

DFL questions should be directed to the appropriate locations, as we cannot advise.

My word is final

User avatar
Linhaohoward
Posts: 23
Joined: Sat Dec 21, 2019 1:23 pm
Has thanked: 3 times

Re: Hardware best practices

Post by Linhaohoward »

torzdf wrote: Sun Dec 22, 2019 10:43 pm

This is the forum for faceswap, not DFL.

DFL questions should be directed to the appropriate locations, as we cannot advise.

Sorry my bad I’m actually intending on changing to Faceswap. Since dfl doesn’t support multi gpu in the first place.

Just hoping to get my hardwares right once and for all so been asking a lot of questions

User avatar
bryanlyon
Site Admin
Posts: 793
Joined: Fri Jul 12, 2019 12:49 am
Answers: 44
Location: San Francisco
Has thanked: 4 times
Been thanked: 218 times
Contact:

Re: Hardware best practices

Post by bryanlyon »

For Faceswap i recommend the multiple smaller gpus and a mid/high end cpu. Ryzen 5/7 and 1-4 2060 supers is a great setup.

User avatar
Linhaohoward
Posts: 23
Joined: Sat Dec 21, 2019 1:23 pm
Has thanked: 3 times

Re: Hardware best practices

Post by Linhaohoward »

bryanlyon wrote: Mon Dec 23, 2019 6:03 pm

For Faceswap i recommend the multiple smaller gpus and a mid/high end cpu. Ryzen 5/7 and 1-4 2060 supers is a great setup.

Thank you for all the tips. Just made the purchase of a Ryzen 5 with 2x2070 Super and nvlink bridge. Im happy now though my credit card is crying 😂 does Faceswap support nvlink?

User avatar
Linhaohoward
Posts: 23
Joined: Sat Dec 21, 2019 1:23 pm
Has thanked: 3 times

Re: Hardware best practices

Post by Linhaohoward »

Just switched to Faceswap yesterday. Is there any difference for multi gpu with nvlink on Faceswap? As compared to just multi gpu without nvlink.

User avatar
bryanlyon
Site Admin
Posts: 793
Joined: Fri Jul 12, 2019 12:49 am
Answers: 44
Location: San Francisco
Has thanked: 4 times
Been thanked: 218 times
Contact:

Re: Hardware best practices

Post by bryanlyon »

No difference. The software will autodetect if NVLink is available and use it. There is no difference in operation, just a small speedup when using multiple GPU.

User avatar
SmellyCheese
Posts: 11
Joined: Wed Jan 08, 2020 3:35 pm
Has thanked: 7 times

Re: Hardware best practices

Post by SmellyCheese »

I have an Nvidia GTX 960 (4GB Vram gaming edition). I am experiencing slow iteration times on larger models and have to use both mem. saving gradients and optimizer savings. I am brutally aware that this card is underpowered by today's standards. Would upgrading to a RTX 2060 super with an AMD FX 8350 (already have) significantly improve training speed?

P.s. - Thanks for all the advice admins. You're great at explaining things to code illiterate people.

The pursuit of knowledge and competence

User avatar
bryanlyon
Site Admin
Posts: 793
Joined: Fri Jul 12, 2019 12:49 am
Answers: 44
Location: San Francisco
Has thanked: 4 times
Been thanked: 218 times
Contact:

Re: Hardware best practices

Post by bryanlyon »

Yes. a 2060 super would let you turn off a lot of the memory preservation you're currently using which is honestly the thing most slowing you down with your 960. I'm not going to promise double the speed, but you'll see some strong speed improvements.

The only problem is that with a 2060 super, your CPU will definitely be the bottleneck. While it's an 8 core CPU, it shares it's FPUs between cores and it may struggle to keep the 2060 super running optimally.

User avatar
SmellyCheese
Posts: 11
Joined: Wed Jan 08, 2020 3:35 pm
Has thanked: 7 times

Re: Hardware best practices

Post by SmellyCheese »

Thanks for the advice! Sorry if this is redundant but - given how training is mostly GPU reliant, keeping my current CPU shouldn't affect it too much, no? I don't do any intense gaming stuff, where I'd imagine upgrading the CPU to match the GPU is non negotiable. If it is paramount, I totally could save a few paychecks and get a Ryzen 3700x

The pursuit of knowledge and competence

User avatar
bryanlyon
Site Admin
Posts: 793
Joined: Fri Jul 12, 2019 12:49 am
Answers: 44
Location: San Francisco
Has thanked: 4 times
Been thanked: 218 times
Contact:

Re: Hardware best practices

Post by bryanlyon »

It's not critical. You could certainly get away without upgrading the CPU, but you WILL see better speeds if you upgrade it later. Might be worth scheduling it sometime down the line.

User avatar
Now
Posts: 10
Joined: Sat Jan 04, 2020 6:30 pm
Has thanked: 6 times
Been thanked: 1 time

Re: Hardware best practices

Post by Now »

Bryan, you mentioned " Compute Tasks " in your post. Should I change the default settings on an RX480 to "Compute" rather than "Graphics" to increase speed ? Any other GPU settings I should change to maximise speed ?

Review of the resource use in the Radeon software shows 100% GPU use, only 3% VRAM use, 10-15% of CPU use (i7) and 33% of regular RAM (32GB) with multiple browsing windows on top of the calculations going on in the background. I'm getting around 2 iterations per second with a batch of 64 (I did read that you suggested a max batch of 20 so will be testing that next time) Would it be possible to configure in some way to use 98% of the VRAM, RAM and processor to speed up the process ? I'd be using a spare laptop during the process.

Appreciate any tips !

User avatar
JohnnyMacIII
Posts: 1
Joined: Thu Feb 27, 2020 4:08 am
Has thanked: 2 times

Re: Hardware best practices

Post by JohnnyMacIII »

Thanks for the write up. How much RAM would you recommend? How much of a performance upgrade would it be from a 3700 to a 3900 if running two rtx 2070 supers? Thanks!

John

User avatar
bryanlyon
Site Admin
Posts: 793
Joined: Fri Jul 12, 2019 12:49 am
Answers: 44
Location: San Francisco
Has thanked: 4 times
Been thanked: 218 times
Contact:

Re: Hardware best practices

Post by bryanlyon »

I'd say stick to a minimum of 16gb. In your case with 2 2070 supers you might want to go up to 32gb, but it's not required.

3700 and 3900 will be pretty similar since they're both more than decent chips and more than you strictly need. Just go with the chip that meets your other needs the best.

Post Reply