[Guide] Vast.ai setup script

Want to use Faceswap in The Cloud? This is not directly supported by the Devs, but you may find community support here


Forum rules

Read the FAQs and search the forum before posting a new topic.

NB: The Devs do not directly support using Cloud based services, but you can find community support here.

Please mark any answers that fixed your problems so others can find the solutions.

Post Reply
User avatar
zzz
Posts: 3
Joined: Sat Feb 22, 2020 4:10 pm
Has thanked: 3 times
Been thanked: 5 times

[Guide] Vast.ai setup script

Post by zzz »

Had some issues with my home computer so used vast.ai to rent instances. A GTX 1080-TI runs about 0.1$/hour.

When creating instances use the tensorflow/tensorflow:1.15.2-gpu-py3-jupyter image

Here's the script I run to setup the enviroment, put it in a .sh file. Open a terminal and do a "chmod +x" on the file and then run the file. Zip is not necessary but useful when downloading the model.

Upload faces and alignment files.

Code: Select all

#!/bin/bash

apt update -y
apt-get install -y zip
apt-get install -y git
apt-get install -y libsm6 libxext6 libxrender-dev 
apt-get install -y libgl1-mesa-dev
git clone -b r1.0 https://github.com/deepfakes/faceswap.git
cd faceswap
pip install -U pip
pip install -r requirements_nvidia.txt
python faceswap.py

Run "python faceswap/faceswap.py" once to setup faceswap for NVIDIA and then run the training.
Edit faceswap/config/training.ini to setup which mask to use and other parameters.

Here's my example training command

Code: Select all

python ./faceswap/faceswap.py train -A ./A -ala ./A.fsa -B ./B -alb ./B.fsa -m ./model -t dfl-h128 -bs 16 -it 1000000 -g 1 -s 100 -ss 25000 -ps 100 -w -ag -L VERBOSE

Check the training_preview.jpg to evaluate progress.

Update: added forced branch r1.0 until I can figure out how to get python3.7 running on vast instances.

Last edited by zzz on Wed Sep 02, 2020 10:07 pm, edited 6 times in total.

Tags:
User avatar
kamapisachihd
Posts: 3
Joined: Sun Jun 07, 2020 12:39 am

Re: [Guide] Vast.ai setup script

Post by kamapisachihd »

First of all thank you for sharing this. I have successfully started training my model on remote GPU with Vast.ai although I have noticed that the GPU capacity of the instance labelled on the site is not what you get. Its actually not even a fraction of the GPU capacity that is being offered. I have RTX 2080 in my rig and I can get decent 9.2 EGs/sec, on the other hand I tried to train the model on Vast.ai and i barely got 150 iterations @ 0.5 EG/sec in 3 hrs of my training time. ( was hiring RTX 2080Ti), i dont know whats going in there but seems like all the systems labelled are place holder names except true processing power on the other end. As much as I like the ease of the site and affordability system rented it meant nothing at the end. I dont know if I am doing something wrong or wondering if you had better experience?

User avatar
bryanlyon
Site Admin
Posts: 793
Joined: Fri Jul 12, 2019 12:49 am
Answers: 44
Location: San Francisco
Has thanked: 4 times
Been thanked: 215 times
Contact:

Re: [Guide] Vast.ai setup script

Post by bryanlyon »

Vast AI is a marketplace, not a single provider. When you buy a system, you're buying from some individual and not a company like Amazon/Google. Some of those people have found ways to "hack" the system to falsify hardware availability and fool Vast's software trying to detect it Report it to Vast and they'll check it and if they find the fake system they'll remove it and generally refund you pretty quickly.

User avatar
police.bike
Posts: 22
Joined: Tue Jun 30, 2020 3:37 pm
Has thanked: 7 times
Been thanked: 5 times

Re: [Guide] Vast.ai setup script

Post by police.bike »

zzz wrote: Wed May 27, 2020 12:19 pm

Had some issues with my home computer so used vast.ai to rent instances. A GTX 1080-TI runs about 0.1$/hour.

When creating instances use the tensorflow/tensorflow:1.15.2-gpu image

Here's the script I run to setup the enviroment, put it in a .sh file. Open a terminal and do a "chmod +x" on the file and then run the file. Zip is not necessary but useful when downloading the model.

Upload faces and alignment files.

Code: Select all

#!/bin/bash

pip install -U pip
apt-get install -y zip
apt-get install -y git
apt-get install -y libsm6 libxext6 libxrender-dev
git clone https://github.com/deepfakes/faceswap.git
cd faceswap
pip install -r requirements.txt

Run "python faceswap/faceswap.py" once to setup faceswap for NVIDIA and then run the training.
Edit faceswap/config/training.ini to setup which mask to use and other parameters.

Here's my example training command

Code: Select all

python ./faceswap/faceswap.py train -A ./A -ala ./A.fsa -B ./B -alb ./B.fsa -m ./model -t dfl-sae -bs 32 -it 1000000 -g 1 -s 100 -ss 25000 -ps 75 -w -ag -L INFO

Check the training_preview.jpg to evaluate progress.

excellent site and ultra easy. thank you !

User avatar
Bolos
Posts: 12
Joined: Thu Jun 25, 2020 1:53 pm
Has thanked: 11 times

Re: [Guide] Vast.ai setup script

Post by Bolos »

Wow! Thanks for the link! I'm stuck with my trash can and it's amd fire pros... And recently have found trouble getting access to google colab's gpu, so this sounds like a afordable way to try getting better at deepfakes without having to spend money on an extra pc :)

I would have one question though, there are lot's of dual or even more cards avaible for some renters. I know faceswap can work with multiple cards, but could anyone here confirm that it is also the case one the cloud computing?

Wishing you all happy deep fakes :)

User avatar
police.bike
Posts: 22
Joined: Tue Jun 30, 2020 3:37 pm
Has thanked: 7 times
Been thanked: 5 times

Re: [Guide] Vast.ai setup script

Post by police.bike »

I think this isn't working anymore.

While installing Git, it forces a apt update and otherwise git won't install.

Once updated, a faceswap clone and nvidia installation is done. But when I try to set backend by running python faceswap.py I get an error saying python 3.7 is required.

I installed it using apt-get install python3.7

But no luck.. stuck with same error. Tried gpu builds on vast like 1.15.2-gpu-py3-jupyter , latest and nightly. Same results... Any help or revised guide for vast ????

User avatar
bryanlyon
Site Admin
Posts: 793
Joined: Fri Jul 12, 2019 12:49 am
Answers: 44
Location: San Francisco
Has thanked: 4 times
Been thanked: 215 times
Contact:

Re: [Guide] Vast.ai setup script

Post by bryanlyon »

If you install python 3.7 using apt-get you may have to call it with python3 instead. Though I don't have time to test for sure.

User avatar
police.bike
Posts: 22
Joined: Tue Jun 30, 2020 3:37 pm
Has thanked: 7 times
Been thanked: 5 times

Re: [Guide] Vast.ai setup script

Post by police.bike »

no luck. Seems faceswap got updated recently and the clone creates this error. Any help is much appreciated

User avatar
bryanlyon
Site Admin
Posts: 793
Joined: Fri Jul 12, 2019 12:49 am
Answers: 44
Location: San Francisco
Has thanked: 4 times
Been thanked: 215 times
Contact:

Re: [Guide] Vast.ai setup script

Post by bryanlyon »

Faceswap has been updated and due to the update newer versions are required. We do still have the previous version available and you can use if you want to use that. Just add

Code: Select all

git checkout r1.0

after the git clone to get the 1.0 version which should work with the existing scripts, but I HIGHLY recommend spending the time to get the 2.0 version working instead.

User avatar
police.bike
Posts: 22
Joined: Tue Jun 30, 2020 3:37 pm
Has thanked: 7 times
Been thanked: 5 times

Re: [Guide] Vast.ai setup script

Post by police.bike »

no luck even after switching to r1.0
I tried with 1.15.2-gpu-python3-jupyter and same image with latest

User avatar
police.bike
Posts: 22
Joined: Tue Jun 30, 2020 3:37 pm
Has thanked: 7 times
Been thanked: 5 times

Re: [Guide] Vast.ai setup script

Post by police.bike »

Can you add the below line to the guide :

apt-get install -y libgl1-mesa-dev

must be done if faceswap errors out. Its something to do with libGL 1.0
This fixed my problems and was able to use vast server on 1.15.2-gpu-py3-jupyter

User avatar
zzz
Posts: 3
Joined: Sat Feb 22, 2020 4:10 pm
Has thanked: 3 times
Been thanked: 5 times

Re: [Guide] Vast.ai setup script

Post by zzz »

police.bike wrote: Sat Aug 29, 2020 7:58 pm

Can you add the below line to the guide :

apt-get install -y libgl1-mesa-dev

must be done if faceswap errors out. Its something to do with libGL 1.0
This fixed my problems and was able to use vast server on 1.15.2-gpu-py3-jupyter

Updated guide with r1.0 branch and this fix.

User avatar
FuorissimoX
Posts: 56
Joined: Mon Sep 21, 2020 6:49 am
Location: Italy
Has thanked: 10 times
Been thanked: 2 times

Re: [Guide] Vast.ai setup script

Post by FuorissimoX »

This procedure allow use gui? Im looking for alternative to Google Cloud because i want try different gpu and google allow only Tesla

User avatar
police.bike
Posts: 22
Joined: Tue Jun 30, 2020 3:37 pm
Has thanked: 7 times
Been thanked: 5 times

Re: [Guide] Vast.ai setup script

Post by police.bike »

command line will be faster and better. But yes, you can have it traditional way using a desktop config onto your remote

Last edited by police.bike on Wed Nov 18, 2020 1:05 am, edited 1 time in total.
User avatar
police.bike
Posts: 22
Joined: Tue Jun 30, 2020 3:37 pm
Has thanked: 7 times
Been thanked: 5 times

Re: [Guide] Vast.ai setup script

Post by police.bike »

FuorissimoX wrote: Wed Sep 23, 2020 8:50 pm

This procedure allow use gui? Im looking for alternative to Google Cloud because i want try different gpu and google allow only Tesla

You can use gnome panel and a bunch of utilities using a initial shell connection to your vm. Install the terminal, xfce and other goodies that you need for desktop access.

Configure your init to start the gnome panel or xfce equivalent at start up
create a vncserver

use putty and output to localhost:5901
This will help you get the desktop access

Install fswap as typical windows or SH installer.

Top

Post Reply