Page 2 of 4
Re: [Resource] Training Using Google Colab
Posted: Sun Dec 13, 2020 1:10 am
by Korben
manintan wrote: ↑Mon Nov 23, 2020 11:52 am
This doesnt even work, many of the steps are outdated or wrong.
If anyone has gotten this to work, please post a working version with all steps and folder naming etc...
This is a cleaned up version of what I am using.
Code: Select all
# -*- coding: utf-8 -*-
"""# Setup Faceswap"""
#@title Set Time Zone to eastern
!rm /etc/localtime
!ln -s /usr/share/zoneinfo/America/New_York /etc/localtime
!date
#above is for HST, you can find yours in
#/usr/share/zoneinfo
#@title Mount Google Drive
from google.colab import drive
drive.mount('/content/drive', force_remount=True)
#@title Download training data
!rm -r face_a
!rm -r face_b
#under the root folder of my google drive ("My Drive") I created a /colab_files/faceswap/faces
#folder that contains zipped face data. Important the zip does not contain folders only the
#faces and alignments files.
#this copies the zip files from your google drive to the workspace while renaming them to a generic name
!cp "/content/drive/My Drive/colab_files/faceswap/faces/face_a_SomeName.zip" "./face_a.zip"
!cp "/content/drive/My Drive/colab_files/faceswap/faces/face_b_SomeName.zip" "./face_b.zip"
#unzip them in folders
!unzip face_a.zip -d face_a
!unzip face_b.zip -d face_b
#delete the zip files
!rm face_a.zip
!rm face_b.zip
#@title Grab the V1 version of Faceswap
!git clone --single-branch --branch r1.0 https://github.com/deepfakes/faceswap.git
#@title Install Tensorflow
#ignore the errors it still works
!pip install -r faceswap/requirements_nvidia.txt
#@title Copy configuration files
#if you copied config files from your pc to your google drive and want to use them uncomment this section.
#!cp "/content/drive/My Drive/colab_files/faceswap/config/train.ini" faceswap/config/
#!cp "/content/drive/My Drive/colab_files/faceswap/config/.faceswap" faceswap/config/
#!ls -lA faceswap/config/
#!cat faceswap/config/train.ini
"""# Run Training"""
#set variables start, will varry according to your trainer type etc...
num_iterations = "3000000"
save_every = "300"
save_model_every = "15000"
batch_num = "48"
num_gpus = "1"
trainer_type = "dfl-h128"
model_dir = "/content/drive/My Drive/colab_files/faceswap/models/YourModelFolder"
alignments_file_a = "face_a/alignments.fsa"
alignments_file_b = "face_b/alignments.fsa"
#set variables end
!python3 faceswap/faceswap.py train \
-A 'face_a' -ala '{alignments_file_a}' \
-B 'face_b' -alb '{alignments_file_b}' \
-m '{model_dir}' \
-t '{trainer_type}' \
-bs '{batch_num}' \
-it '{num_iterations}' \
-g '{num_gpus}' \
-s '{save_every}' \
-ss '{save_model_every}' \
Re: [Resource] Training Using Google Colab
Posted: Tue Feb 23, 2021 5:43 am
by police.bike
Update - Feb 20, 2021
I was able to get both python 3.7 & 3.8 working on google collab and train using latest versions. Pretty amazing ones !
Just use this as block 1 before running any steps within the Collab notebook
Code: Select all
!sudo apt install python3.7
!update-alternatives --install /usr/local/bin/python3 python3 /usr/bin/python3.6 1
!update-alternatives --install /usr/local/bin/python3 python3 /usr/bin/python3.7 2
!python --version
!sudo apt update && upgrade
!sudo apt install python3-pip
!python -m pip install --upgrade pip
!pip --version
!python --version
!pip --version
Change 3.7 to 3.8 if you need it. This will reset python runtime to 3.7.5 and will get latest faceswap editions working.
Have fun !!!
Re: [Resource] Training Using Google Colab
Posted: Wed Feb 24, 2021 6:47 am
by oNafodo
I put that all in a notebook and attempted to run it, all the steps worked except the final one of actually running the training.
This is the final cell I used
Code: Select all
"""# Run Training"""
#set variables start, will varry according to your trainer type etc...
num_iterations = "3000000"
save_every = "300"
save_model_every = "15000"
batch_num = "48"
num_gpus = "1"
trainer_type = "dfl-h128"
model_dir = "/content/drive/My Drive/colab_files/faceswap/models/Olya"
alignments_file_a = "face_a/yt1s.fsa"
alignments_file_b = "face_b/Olya.fsa"
#set variables end
!python3 faceswap/faceswap.py train \
-A 'face_a' -ala '{alignments_file_a}' \
-B 'face_b' -alb '{alignments_file_b}' \
-m '{model_dir}' \
-t '{trainer_type}' \
-bs '{batch_num}' \
-it '{num_iterations}' \
-g '{num_gpus}' \
-s '{save_every}' \
-ss '{save_model_every}' \
But when I run it, this is what I get back.
Code: Select all
Setting Faceswap backend to NVIDIA
usage: faceswap.py [-h] {extract,train,convert,gui} ...
positional arguments:
{extract,train,convert,gui}
extract Extract the faces from pictures or a video
train Train a model for the two faces A and B
convert Convert source pictures or video to a new one with the
face swapped
gui Launch the Faceswap Graphical User Interface
optional arguments:
-h, --help show this help message and exit
faceswap.py: error: unrecognized arguments: 1
Is there something I am supposed to be changing that I didn't?
Re: [Resource] Training Using Google Colab
Posted: Wed Mar 03, 2021 6:59 pm
by hereforfun
oNafodo wrote: ↑Wed Feb 24, 2021 6:47 am
I put that all in a notebook and attempted to run it, all the steps worked except the final one of actually running the training.
But when I run it, this is what I get back.
Code: Select all
Setting Faceswap backend to NVIDIA
usage: faceswap.py [-h] {extract,train,convert,gui} ...
positional arguments:
{extract,train,convert,gui}
extract Extract the faces from pictures or a video
train Train a model for the two faces A and B
convert Convert source pictures or video to a new one with the
face swapped
gui Launch the Faceswap Graphical User Interface
optional arguments:
-h, --help show this help message and exit
faceswap.py: error: unrecognized arguments: 1
Had the same issue, just remove the line containing the -g flag.
Re: [Resource] Training Using Google Colab
Posted: Fri Mar 05, 2021 6:56 am
by oNafodo
Thanks a million, that sorted it out.
Re: [Resource] Training Using Google Colab
Posted: Tue Mar 16, 2021 6:04 pm
by zachdidit
I've got a Collab working with the latest version of Faceswap. Also added a cell to keep the runtime open through idle.
Be sure to change folders, filenames, timezone, etc
https://colab.research.google.com/drive ... sp=sharing
Re: [Resource] Training Using Google Colab
Posted: Tue Apr 13, 2021 12:26 am
by HoloByteus
Spent a couple days learning colab and using faceswap with it using the script in this post which saved a ton of time. I'm getting an error on training though which I can't seem to figure out. I have the alignment files in the face_a and b folders which are under /content but it doesn't seem to like the alignment arguments. Any ideas?
This is the error:
faceswap.py: error: unrecognized arguments: -ala face_a/alignments.fsa -alb face_b/alignments.fsa
And my training code.
Code: Select all
#set variables start
num_iterations = "120000"
save_every = "360"
save_model_every = "25000"
batch_num = "16"
num_gpus = "1"
trainer_type = "villain"
model_dir = "/content/drive/MyDrive/colab_files/faceswap/models/ModelAB"
alignments_file_a = "face_a/alignments.fsa"
alignments_file_b = "face_b/alignments.fsa"
timelapse_dir = "/content/drive/MyDrive/colab_files/faceswap/output/timelapse"
#set variables end
!python3 faceswap/faceswap.py train \
-A 'face_a' -ala '{alignments_file_a}' \
-B 'face_b' -alb '{alignments_file_b}' \
-m '{model_dir}' \
-t '{trainer_type}' \
-bs '{batch_num}' \
-it '{num_iterations}' \
-s '{save_every}' \
-ss '{save_model_every}' \
-tia 'face_a' \
-tib 'face_b' \
-to '{timelapse_dir}'
I did notices one thing wierd. i have about 7000 images in face_a and scrolling though the list I can't see the alignment file, you see a ... and no way to expand it. I tried to upload a copy of the alignment file and move it to the folder and it fails. My storage isn't full. If I can't see it perhaps neither can the scripts. Is there a limit on number of files in a folder?
Re: [Resource] Training Using Google Colab
Posted: Tue Apr 13, 2021 9:14 pm
by HoloByteus
Getting the following with Tensorflow install.
ERROR: albumentations 0.1.12 has requirement imgaug<0.2.7,>=0.2.5, but you'll have imgaug 0.2.9 which is incompatible.
WARNING: The following packages were previously imported in this runtime:
[psutil]
You must restart the runtime in order to use newly installed versions.
Re: [Resource] Training Using Google Colab
Posted: Tue Apr 13, 2021 11:14 pm
by sp13
I get the same error message about albumentations but it doesn't seem to hurt anything so I just ignore it. You do have to press the button to restart the runtime though (or at least I always do it.)
One thing to note is that Colab comes with Tensorflow already installed and Google advises you not to install your own because theirs is optimized for their hardware. So you shouldn't install another if you can get away with it. However they currently use Tensorflow 2.4 and I use 2.2 locally, so I have to downgrade theirs or I can't use the models it produces.
About your previous question, training doesn't need alignment files anymore so you shouldn't set those options. The necessary data is stored right in the face set image files, provided that you have extracted them with a recent version of faceswap.
Re: [Resource] Training Using Google Colab
Posted: Tue Apr 13, 2021 11:58 pm
by HoloByteus
Thank you. I was thinking something with the colab required the alignment files but doh, yea I guess the notebook was made before that change so removed them from the training cell resolved that issue.
Figured out the psutil issue, I was running the GPU check (which includes a psutil install) before installing Tensorflow. Not doing that resolved it. Looks like Villian requires Tensorflow install and upon restort ... TRAINING NOW WORKS!
The master notebook needs a few updates. I also changed the Data transfer to point to the content directory and then just zip the extracted faces folders and have the unzip create the folders on colab. Bit easier zip process.
Code: Select all
#@title Download training data
!unzip "/content/drive/MyDrive/colab_files/faceswap/faces/face_a.zip" -d /content
!unzip "/content/drive/MyDrive/colab_files/faceswap/faces/face_b.zip" -d /content
Re: [Resource] Training Using Google Colab
Posted: Wed Apr 14, 2021 2:13 am
by sp13
You should be able to unzip them without even copying them. I imagine something like
Code: Select all
!unzip "/content/drive/MyDrive/colab_files/faceswap/faces/face_a.zip" -d /content/
should work, though I use tar instead of zip.
Also for GPU check, I like to throw in
Code: Select all
!nvidia-smi
import tensorflow as tf
print("Tensorflow version: " + tf.__version__)
which will tell you which GPU and Tensorflow version you have.
Re: [Resource] Training Using Google Colab
Posted: Thu Apr 15, 2021 2:17 am
by HoloByteus
Good point, will make the change to data transfer.
What are you using to bypass the colab interactive timeout? I have the javascript to click the connect button but I'm not a fan of how that keeps opening the cpu/gpu graphs. I found some javascript to click the comments button instead but would like to click the favorite Star instead for minimal visual impact. I haven't tested it yet but was wondering if this would work guessing the star for favorites would be labeled star.
Code: Select all
function ClickConnect(){
console.log("Working");
document.querySelector("#star > span").click()
}
setInterval(ClickConnect,5000)
Re: [Resource] Training Using Google Colab
Posted: Thu Apr 15, 2021 4:59 pm
by sp13
HoloByteus wrote: ↑Thu Apr 15, 2021 2:17 am
What are you using to bypass the colab interactive timeout?
I use some Javascript snippet that I'm not sure actually works because I've gotten timed out before and sometimes get a "Are you still here?" captcha.
I ran across this stackoverflow the other day with lots of things to try, though i haven't done so
https://stackoverflow.com/questions/571 ... connecting
Re: [Resource] Training Using Google Colab
Posted: Thu Apr 15, 2021 5:29 pm
by HoloByteus
Yes, found that same thread and decided on the following since it has a method to enable/disable. Changed it to target the comments instead of connect and that seemed to work enough to pass the first level check for interactivity. Carried me all the way through the hours I had left of the 12hr timeout after jumping through all the hoops to get this running.
Code: Select all
var startClickConnect = function startClickConnect(){
var clickConnect = function clickConnect(){
console.log("Connnect Clicked - Start");
document.querySelector("#comments > span").click();
console.log("Connnect Clicked - End");
};
var intervalId = setInterval(clickConnect, 60000);
var stopClickConnectHandler = function stopClickConnect() {
console.log("Connnect Clicked Stopped - Start");
clearInterval(intervalId);
console.log("Connnect Clicked Stopped - End");
};
return stopClickConnectHandler;
};
var stopClickConnect = startClickConnect();
Re: [Resource] Training Using Google Colab
Posted: Thu Apr 15, 2021 5:31 pm
by sp13
Another random tip that I don't think is in this thread:
If you put the following line somewhere after the git clone that installs faceswap
Code: Select all
!echo "{\"backend\": \"nvidia\"}" > faceswap/config/.faceswap
then faceswap will know that you want to use nvidia and won't ask you about it when you start training.
Re: [Resource] Training Using Google Colab
Posted: Fri Apr 16, 2021 12:45 am
by HoloByteus
Yes, I copied the .faceswap from my local install to the config folder in drive and copy it and the training.ini.
Re: [Resource] Training Using Google Colab
Posted: Wed May 26, 2021 1:35 pm
by awelmisin
hey all, i just started doing this and having an error.. all steps are ok until final one.
i cant seem to use training, and dont know how to fix. i tried to delete something, edit something but nothing worked.
the code i'm using for training is
Code: Select all
#set variables start
num_iterations = "150000"
save_every = "360"
save_model_every = "25000"
batch_num = "16"
num_gpus = "1"
trainer_type = "villain"
model_dir = "/content/drive/My Drive/colab_files/faceswap/models/denisea"
alignments_file_a = "face_a/2_alignments.fsa"
alignments_file_b = "face_b/alignments.fsa"
timelapse_dir = "/content/drive/My Drive/colab_files/faceswap/output/timelapse"
#set variables end
!python3 faceswap/faceswap.py train \
-A 'face_a' -ala '{alignments_file_a}' \
-B 'face_b' -alb '{alignments_file_b}' \
-m '{model_dir}' \
-t '{trainer_type}' \
-bs '{batch_num}' \
-it '{num_iterations}' \
-s '{save_every}' \
-ss '{save_model_every}' \
-tia 'face_a' \
-tib 'face_b' \
-to '{timelapse_dir}'
and the error i'm getting is,
Code: Select all
Setting Faceswap backend to NVIDIA
usage: faceswap.py [-h] {extract,train,convert,gui} ...
positional arguments:
{extract,train,convert,gui}
extract Extract the faces from pictures or a video
train Train a model for the two faces A and B
convert Convert source pictures or video to a new one with the
face swapped
gui Launch the Faceswap Graphical User Interface
optional arguments:
-h, --help show this help message and exit
faceswap.py: error: unrecognized arguments: -ala face_a/2_alignments.fsa -alb face_b/alignments.fsa
Re: [Resource] Training Using Google Colab
Posted: Wed May 26, 2021 8:24 pm
by foundmyway89
My Colab setup has been working great for a few months, but just started receiving an error yesterday. I recreated the notebook in a different environment with a smaller dataset and still get the same error. Has there been an update involving tensorflow that may be causing this? My notebook is based on the one linked in this thread and I am not seeing any errors on the setup steps before running.
Code: Select all
First time configuration. Please select the required backend
1: AMD, 2: CPU, 3: NVIDIA: 3
Faceswap config written to: /content/faceswap/config/.faceswap
Setting Faceswap backend to NVIDIA
05/26/2021 10:18:35 INFO Log level set to: INFO
Using TensorFlow backend.
05/26/2021 10:18:39 ERROR Got Exception on main handler:
Traceback (most recent call last):
File "/content/faceswap/lib/cli/launcher.py", line 153, in execute_script
script = self._import_script()
File "/content/faceswap/lib/cli/launcher.py", line 43, in _import_script
module = import_module(mod)
File "/usr/lib/python3.7/importlib/__init__.py", line 127, in import_module
return _bootstrap._gcd_import(name[level:], package, level)
File "<frozen importlib._bootstrap>", line 1006, in _gcd_import
File "<frozen importlib._bootstrap>", line 983, in _find_and_load
File "<frozen importlib._bootstrap>", line 967, in _find_and_load_unlocked
File "<frozen importlib._bootstrap>", line 677, in _load_unlocked
File "<frozen importlib._bootstrap_external>", line 728, in exec_module
File "<frozen importlib._bootstrap>", line 219, in _call_with_frames_removed
File "/content/faceswap/scripts/train.py", line 13, in <module>
from keras.backend.tensorflow_backend import set_session
File "/usr/local/lib/python3.7/dist-packages/keras/__init__.py", line 3, in <module>
from . import utils
File "/usr/local/lib/python3.7/dist-packages/keras/utils/__init__.py", line 27, in <module>
from .multi_gpu_utils import multi_gpu_model
File "/usr/local/lib/python3.7/dist-packages/keras/utils/multi_gpu_utils.py", line 7, in <module>
from ..layers.merge import concatenate
File "/usr/local/lib/python3.7/dist-packages/keras/layers/__init__.py", line 4, in <module>
from ..engine.base_layer import Layer
File "/usr/local/lib/python3.7/dist-packages/keras/engine/__init__.py", line 3, in <module>
from .input_layer import Input
File "/usr/local/lib/python3.7/dist-packages/keras/engine/input_layer.py", line 7, in <module>
from .base_layer import Layer
File "/usr/local/lib/python3.7/dist-packages/keras/engine/base_layer.py", line 12, in <module>
from .. import initializers
File "/usr/local/lib/python3.7/dist-packages/keras/initializers/__init__.py", line 124, in <module>
populate_deserializable_objects()
File "/usr/local/lib/python3.7/dist-packages/keras/initializers/__init__.py", line 49, in populate_deserializable_objects
LOCAL.GENERATED_WITH_V2 = tf.__internal__.tf2.enabled()
File "/usr/local/lib/python3.7/dist-packages/tensorflow_core/python/util/module_wrapper.py", line 193, in __getattr__
attr = getattr(self._tfmw_wrapped_module, name)
AttributeError: module 'tensorflow._api.v1.compat.v2' has no attribute '__internal__'
05/26/2021 10:18:39 CRITICAL An unexpected crash has occurred. Crash report written to '/content/faceswap/crash_report.2021.05.26.101838178504.log'. You MUST provide this file if seeking assistance. Please verify you are running the latest version of faceswap before reporting
Re: [Resource] Training Using Google Colab
Posted: Wed May 26, 2021 9:26 pm
by torzdf
awelmisin wrote: ↑Wed May 26, 2021 1:35 pm
hey all, i just started doing this and having an error.. all steps are ok until final one.
i cant seem to use training, and dont know how to fix. i tried to delete something, edit something but nothing worked.
Code: Select all
faceswap.py: error: unrecognized arguments: -ala face_a/2_alignments.fsa -alb face_b/alignments.fsa
Remove the alignments A and B options. They are no longer required.
Re: [Resource] Training Using Google Colab
Posted: Wed May 26, 2021 11:33 pm
by awelmisin
https://github.com/awelmisin/faceswap-google-colab
can find the latest working version here, i am also taking a youtube video about this.