Original Model can still produce great results!

Display post links to interesting swaps


Locked
User avatar
cosmico
Posts: 95
Joined: Sat Jan 18, 2020 6:32 pm
Has thanked: 13 times
Been thanked: 34 times

Original Model can still produce great results!

Post by cosmico »

One of the biggest misconceptions I think the community has is wanting higher quality results, but not using
higher quality data, or getting impatient and not training their models to completion. Instead they just search for a better model. Sometimes people who are unhappy with their results will take on ambitious projects using large heavy models with unreasonable large data sets, and the end result is that they have no hope of ever completing because of how slow these models will be. I know I still do this sometimes, and I assume you probably do as well. So in order to examine this issue in a different light, I thought I'd show that the "bad" models can give amazing results, and what better way than using original model trainer, and try to do a great deepfake. Hopefully the take away is that while yes, "good" models can produce better results than "bad" models, "bad" models can still produce good results!

For best results Keep it in the small window, but make sure to turn on 1080p

.
UPDATE VIDEO I was encouraged in the comments below to release the model and I decided to do a little more training on it before releasing. Here is the updated results. It's hard to notice the difference unless you overlay the old converted video over the new one, but it is better. Once again for best appearance, make the video 1080p but don't maximize the video

.
.
I understand why people have a hard time with the patience aspect, and not getting there models to maximum training,
It takes such a long time. And depending on your set up, faceswap may take up so many resources
that you won't be able to do most other things on your computer while it's training. It's almost like going on a computer fast. And one of the biggest reasons that discourages me from doing this is the possibility of wasting my time. "What If I let it train for several weeks and it hits max training, but even though it can't train any more detail...it still looks off.... perhaps because their facial shapes are slightly different and it looks weird" or perhaps "I knew there hair color was different when I started but I didn't think it would make that big of a deal, but it clearly does, and this feels off". It's these thoughts of wasting my time while doing this training computer fast that discourages me from having patience in the training process and I assume it probably affects you too. I was thinking about ways I could combat this thinking in myself when I had an idea that might help others out.

So I had an idea.
If you want to practice, You can practice on my exact deepfake.
I was reading user JansenSensei post in the general discussion asking for a "an online database full of datasets of celebrities known to give good results "basically eliminating the part of the faceswap where you search for data. The idea of it being if you wanted to do a Trump Faceswap for your next project, you could just go to a website and download a "trump pack" and you would be good to go. Sounds like a great idea, but the mods informed them why it wouldn't work. Essentially, it's because you want your data set to specifically correspond with your other data set for the highest quality possible swap. But you know what would work? Giving people an "A" and "B" data set, and knowing (with a very good estimate)at what the end product "Converted Video C" should look like. And it occurred to me. This could be an excellent learning experience for those interested in practicing! Someone might want to try and make a faceswap for the first time, but even though they've memorized the guides by heart, they don't know where to start, what's too ambitious for them, and what results they should expect.

Initially the idea of someone making the exact same faceswap as one already made seems pointless, But It can properly teach the lesson of high quality data and extensive training being the key ingredients to a quality product because it eliminates the fear that the training will be wasted because the end result will some how end up bad. How does it eliminate this fear you ask? Well if you are going to recreate my swap as practice, you've already seen my final result so you know your time won't be wasted. You know exactly what you are going to get. So basically I'm going to give you both my data sets (in video form), and show you what it's capable of producing if you have the patience. So you know your time won't be wasted.
**Before you start trying it on your own though, you should know there's always some variability in the way the model learns. So even though the end destination of maximum training is always the same, the model
will always take different routes to get there, every single time.

This Video was my A data, but I only used the first 2 minutes of it, and this Video was my B data.

If you want a youtube video downloader, my favorite is simply typing the two letters "pp" into the youtube url right after the word "tube" and before ".com" on the specific video you want, like this "https://www.youtubePP.com/watch?v=iEY07Ut..." and it will take you to a download page for that exact video.

Here are the exact Settings I used:
EXTRACTION
Extractions were 256 (I'm still using the older faceswap. Newer faceswap's default of 512 shouldn't affect anything)
TRAIN
Model: Original
Coverage: 73%
Learning Rate 5.1e-5
Allow Growth
Mexed Precision
Global>LOSS
Loss function: Mae
Mask Loss function: Mae
Penalized mask loss
Eye and mouth multipler: Initially on defaults of 3 and 2, but started to mess around with after 50 hours, never going over a multiplier of 20
mask type extended
The amount of training I did was 160 million examples given
The amount of training time was about 115 hours.\
CONVERSION
Gaussian Sharp of 240, radius .3, threshold 5.0
Writer crf of 14 with very fast preset
My average eg/s was around 400. It trained for 115 hours, and almost 160 million Examples Given

Unless your computer is the same as mine, it won't train at the same speed. Just because you trained for the same amount of hours as me doesn't mean you trained the exact amount. Also You may or may not be able to even reach my batch size. My results shown in the beginning of this post are at 160 million examples given. To count the amount of examples given you trained, I went to the analysis tab and looked at the total time trained and convert how many hours and minutes the model has been training into seconds, then times how many seconds of training with the average examples given per second. Your results should look like mine when you've hit 160 million examples given of training.

Acknowledgements:
-I know I didn't use the highest quality data possible. But it was decently good and I had patience and I still got amazing results. I feel like this only proves the point more that if you want high quality results your best options are high quality data and patience. Because look what I got with decent data and patience.
-I didn't train to to max training, but I felt it was close enough to make the point that original can still produce great results with out giving it another 200 hours of training.
-The video was intentionally cut to exclude out all the times the camera zoomed in Melissa and Margot, it naturally gets worse the bigger the face is. Nothing you can do about that except control for what type of video you choose to convert.
-I think this project has lead me to believe that While you can technically faceswap anyone, unless you are personally recording the video of the person in HD, with amazing lighting, reacting at many different angles; your data/ product will probably be low quality. Realistically there's like 3 scenarios for high quality data/products. Volunteers you record in a profession manner like I just mentioned, Movie and TV stars, and youtubers.

  • In the very beginning I stated that ". Sometimes people who are unhappy with their results will take on ambitious projects using large heavy models with unreasonable large data sets that will be hopeless endless" My source was me. Like... as of this very second. :roll: I'm attempting a villain model and I've already invested 150 hours into it so I cant quit now. Some people will never learn.

If you found this write up or that potential practice session useful, leave a like It makes me Smile :)

Last edited by cosmico on Thu Mar 11, 2021 9:22 pm, edited 2 times in total.
User avatar
ugramund
Posts: 27
Joined: Sat Feb 20, 2021 3:56 am
Location: India
Has thanked: 25 times
Been thanked: 4 times

Re: Original Model can still produce great results!

Post by ugramund »

Thank you for this detailed write up.

I too have always believed that old models like Original & Dfaker can give good results even in 720p if the person's face is at some distance.

I would like to request you that can you share your trained model for others to download because I think a trained model can help in faster training when we compare it to a model that has to be trained from scratch.

I know this theory of pre trained model is debatable but I think even @torzdf has not completely disagreed with that.

I have read some articles on Neural Networks & some works of DFL users who have created good swaps both of which suggested that they pre trained their model with random faces before starting their actual project.

When I 1st started 2 years ago on a swapping software known as FakeApp, it was provided with Trump-Cage model as default. I was then training on laptop using my i5 CPU. But even it quickly recognized the facial features of the image sets of my project in the preview window. It was a swap of a 10 second video which I got after 40 hours of CPU training & was pretty good considering the hardware I used.

Sadly, after 2 years, I am still on my laptop training with faceswap but this time when I started to train using Original model, it took about 3 hours to draw something that we can call as a face in the preview window because it is not a pre trained model & I have to train it from scratch.

So from my past experience, I started a project of creating a pre trained model. For this I made 2 sets. One set had 150 HD images of 150 different people from around the world with different ethnicity, age & gender with different facial expression & lighting conditions.

The second set was of a single celebrity but with 1300+ super HD images with different facial expressions & lighting conditions.

While training, I use only one set at a time on both sides A & B since I want it to get trained in recognizing different faces rather than a model which was trained to perform a single swap.

Till now I have trained for 20 days with batch of 9 for more than 130 hours & it gave me pretty good result for a test that I did on it.

I extracted images from videos of 2 different persons who were not the part of my pre training data set & started training & the preview window instantly recognized their facial features although they were not looking like their original faces but the facial features predicted by the model was accurate & after 3-4 minutes of training, their original faces also came back in the preview window. After that I removed it without creating a swap video & continued my training.

I know it will take a long time to train on CPU & so I am planning to train for a week more & then share it here on this forum so that if someone wants a pre trained Original model, then they can get it from here.

In love with the the RealFace model.

User avatar
cosmico
Posts: 95
Joined: Sat Jan 18, 2020 6:32 pm
Has thanked: 13 times
Been thanked: 34 times

Re: Original Model can still produce great results!

Post by cosmico »

ugramund wrote: Sun Mar 07, 2021 6:50 am

would like to request you that can you share your trained model for others to download because I think a trained model can help in faster training when we compare it to a model that has to be trained from scratch.

I would love to share my Original model, but how am I supposed to upload a 29Gb file for people to download?

User avatar
ugramund
Posts: 27
Joined: Sat Feb 20, 2021 3:56 am
Location: India
Has thanked: 25 times
Been thanked: 4 times

Re: Original Model can still produce great results!

Post by ugramund »

cosmico wrote: Sun Mar 07, 2021 6:14 pm
ugramund wrote: Sun Mar 07, 2021 6:50 am

would like to request you that can you share your trained model for others to download because I think a trained model can help in faster training when we compare it to a model that has to be trained from scratch.

I would love to share my Original model, but how am I supposed to upload a 29Gb file for people to download?

I didn't know that the size of the model files grow. I thought they remain of constant size.
Are you saying that the 2 main files of the model folder original.h5 & original_state.json are of 29GB?

I do know that the original_logs folder size grows but whatever little training I did on my CPU for 77k iterations in the
last 20 days, my 2 main model files original.h5 & original_state.json never changed their file size.

And one more request, please make this YouTube video public from unlisted so that it can appear in the search results and if possible add "Faceswap Original model" in your video title since we already have very few videos of Faceswap on Youtube & yours being such a great swap video should come in search results for people who search for demo Faceswap videos before actually trying it.

In love with the the RealFace model.

User avatar
cosmico
Posts: 95
Joined: Sat Jan 18, 2020 6:32 pm
Has thanked: 13 times
Been thanked: 34 times

Re: Original Model can still produce great results!

Post by cosmico »

Here's the link to the model files. It's about 626mb.

For anyone wanting to download, just don't use them for anything bad or try to take credit for them :D
The Original Model

Also I uploaded a slightly better version of the swap in the original post up top, and I made them both public on youtube.

If you found this helpful, make sure to leave a thanks, It makes me happy :D

User avatar
ugramund
Posts: 27
Joined: Sat Feb 20, 2021 3:56 am
Location: India
Has thanked: 25 times
Been thanked: 4 times

Re: Original Model can still produce great results!

Post by ugramund »

cosmico wrote: Thu Mar 11, 2021 9:17 pm

Here's the link to the model files. It's about 626mb.

For anyone wanting to download, just don't use them for anything bad or try to take credit for them :D
The Original Model

Also I uploaded a slightly better version of the swap in the original post up top, and I made them both public on youtube.

If you found this helpful, make sure to leave a thanks, It makes me happy :D

Thanks buddy for putting the download link & making both the videos public with adding Faceswap in title.
You can also put the model download link in the video description & pin it on comment section on YouTube so that more people can download it.

UPDATE :- All files working fine although very slow for my CPU training may be because you have used Mixed Precision & it gives me a warning message in the log window that it will run slow on PCs without GPU.

By the way, next time you upload a model, only use the "original.h5" & "original_state.json'' files to upload.
The "original.h5.bk" & "original_state.json.bk'' files are backup files for recovery with ".bk" extension & are of no use.

I also make archived ZIP backups of my model using only the "original.h5" & "original_state.json'' files & it has worked
perfectly fine whenever I have extracted & used them.

In love with the the RealFace model.

Locked