Questions about multiple GPUs

Talk about Hardware used for Deep Learning


Locked
User avatar
ericpan0513
Posts: 23
Joined: Wed Jul 22, 2020 3:34 am
Has thanked: 6 times

Questions about multiple GPUs

Post by ericpan0513 »

Hi!
I've just been doing face swap projects recently, and I have some questions about multiple GPUs.

  1. First thing is that, what exactly should I do to run my face swap project on several GPUs since I haven't purchase them but I will in few weeks so that I can't try myself now. Should I just using like"export CUDA_VISIBLE_DEVICES=0,1,2," than it will operate successfully? Or I need to change something in the code.
  2. The second question is that if I want to run one face swap project on multiple devices/computers to speed up a lot, is there a way to do this? Maybe sending data through the internet, but how can I know how the works are assigned to different GPUs? Can someone tell how the GPUs are chosen and how the original codes work?
    Thank you !
    (First time asking, not sure if it's proper. If not, sorry~)
User avatar
torzdf
Posts: 2649
Joined: Fri Jul 12, 2019 12:53 am
Answers: 159
Has thanked: 128 times
Been thanked: 622 times

Re: Questions about multiple GPUs

Post by torzdf »

For training there is a "GPUs" option. Just select the number of GPUs you want to use there, and make sure you set a batchsize (when training) that is divisible by the number of GPUs available.

There is no current way for distributed training over multiple machines, but we have improvements for multi-gpu in the pipeline.

My word is final

User avatar
ericpan0513
Posts: 23
Joined: Wed Jul 22, 2020 3:34 am
Has thanked: 6 times

Re: Questions about multiple GPUs

Post by ericpan0513 »

OK, thanks for answering. It helps a lot!

Locked