Fantastic results from 8 Original Pictures

Want to know about the Faceswap's Face Extraction process? Got tips, ideas or just want to learn about how it all works? Then this is the place for you


Forum rules

Read the FAQs and search the forum before posting a new topic.

This forum is for discussing tips and understanding the process involved for Extracting and preparing face sets for training a model in Faceswap.

If you have found a bug are having issues with the Extraction process not working, then you should post in the Extract Support forum.

Please mark any answers that fixed your problems so others can find the solutions.

Locked
User avatar
363LS2GTO
Posts: 30
Joined: Fri Jul 08, 2022 7:06 pm
Has thanked: 1 time
Been thanked: 4 times

Fantastic results from 8 Original Pictures

Post by 363LS2GTO »

After taking a few months off, I decided to finish my small Model B training set project.

I began with 5 good quality photos showing the front and sides of my model. I also had 3 smaller, low resolution black and white photos.

This was my first swap and it has been a learning process (and still is). I was not sure if it would produce acceptable results as most everyone says it is not possible. I can tell you that with the right amount of time and effort, a quality swap is possible.

I used a variety of programs to enhance and expand my data set.

1)Vance AI Photo colorizer
2)FaceApp Mobile
3)Remini
4)Blender with Klein tools 'head' add on.
5)Topaz Video AI
6)Windows Video editor

I also used a few techniques along the way to increase the data set.

I have performed 3 swaps. Each swap has become progressively better. After each swap I have used my judgement to pick a few of the best (most accurate to the original model) high quality photos to add to the training set. Using Remini and Blender with the Klein Tools package, each new photo can produce several hundred new high quality photos using AI to produce new facial expressions.

For angles that I was originally missing, I found models with similar faces and and used this data on my early swaps, pulling out these models as I acquired better data produced by faceswap.

Along the way I continued to increase the quality of Model A, Model B, and the photos / videos for convert. As others have stated, the importance of high quality data can not be over emphasized.

For my most recent swap I finally found a great Model A head shape match. I used around 5,000 photos for Model A and have accumulated around 3500 high quality photos for Model B (from the original 8).

Running villain up to 850,000 iterations at 136 hours (I will probably keep going to a million) and then passing the video through Topaz AI, the results are better than I ever thought possible with my limited amount of original data.

I have to take into account the best angles and subject distance to the camera in the video used for conversion, but I believe most people have to be careful about this even with the best original data.

Its not perfect, but the swapped model is very accurate. Keeping the vast majority of the data as the original subject with just a slight amount of the best results from my previous faceswaps helped to keep the original Model B true to life.

The details are sharp and the mask motion is smooth and clear (realistic) overall. Some parts of the videos are better than others, but when it is looking good, it looks 85% as good as anything I have seen on the net produced with consumer grade hardware and software. It is simply a high quality, fairly convincing swap and I consider the results worth the monetary investment I made (new RTX 3500, used CPU, software expenses).

Hope this helps someone.

User avatar
torzdf
Posts: 2649
Joined: Fri Jul 12, 2019 12:53 am
Answers: 159
Has thanked: 128 times
Been thanked: 623 times

Re: Fantastic results from 8 Original Pictures

Post by torzdf »

This sounds totally insane, and I'm surprised you got anything out of it!

But it's also interesting stuff, and I'm always keen to see results of people's esoteric experiments when they go outside of the norm. If there are any images you can share, please do, but appreciate that may not be possible.

My word is final

User avatar
haoylee
Posts: 9
Joined: Thu Nov 18, 2021 3:16 pm
Has thanked: 1 time
Been thanked: 1 time

Re: Fantastic results from 8 Original Pictures

Post by haoylee »

From my experience, even if you FEEL comfortable with the results showing on the training preview window, it still wouldn't come out as a fine final video. The glitch, the deformation, etc, are most likely inevitable when produce a sequenced image. Though there might be some perfect still image to select.

User avatar
cedenburn
Posts: 11
Joined: Sun Jan 02, 2022 3:11 am
Has thanked: 11 times

Re: Fantastic results from 8 Original Pictures

Post by cedenburn »

I used My heritage website and their AI to expand a data set from a picture. Still working on it however.

Last edited by cedenburn on Tue May 09, 2023 3:26 pm, edited 2 times in total.
Locked