General Convert

Converting source frames into swapped frames
Forum rules
Read the FAQs and search the forum before posting a new topic.

Please mark any answers that fixed your problems so others can find the solutions.
Post Reply
User avatar
desertskye
Posts: 6
Joined: Thu Aug 06, 2020 2:15 am

General Convert

Post by desertskye »

Hi. I'm still super confused on the whole convert process. I have my video "A" and the alignments file for "A", and a trained model. What do I have to do in conversion to swap Face B onto video A? I'm not seeing anywhere to put it, and am generally confused on the conversion process. Thanks

User avatar
torzdf
Posts: 853
Joined: Fri Jul 12, 2019 12:53 am
Answers: 122
Has thanked: 23 times
Been thanked: 170 times

Re: General Convert

Post by torzdf »

This thread should help, where your specific issue is also covered (a little later on).

if you're still struggling, let us know:

viewtopic.php?f=7&t=728
My word is final

User avatar
desertskye
Posts: 6
Joined: Thu Aug 06, 2020 2:15 am

Re: General Convert

Post by desertskye »

Hey, so ive read through the forum. So the convert process is not for faceswapping (ex Face B onto video A). What process do i run to do a faceswap/deepfake? Thanks

User avatar
desertskye
Posts: 6
Joined: Thu Aug 06, 2020 2:15 am

Re: General Convert

Post by desertskye »

Sorry, I misread the thread. The conversion process is for faceswapping, yet the video that I want to swap on (video b) has no bearing? How would I choose which video to faceswap onto video A? thanks

User avatar
desertskye
Posts: 6
Joined: Thu Aug 06, 2020 2:15 am

Re: General Convert

Post by desertskye »

Sorry for the spam, I'm testing this as I post. I've been running my convert process and its swapping a face onto my original "A" video, but it seems to just be choosing one of the faces I used for training my model. For example, I trained my model with about 10 different faces, but its only using trump (one of the training faces) as the faceswap "B" face. How do I make it use a certain face. Secondly, it seems that I'm swapping the face, but only the face from middle of forehead to chin. Is there a way to swap the whole image including background, where the new face is moving? For now it's swapping trump onto the original video, yet the original video's background and hair is still in the end video. Is this an issue that I can change or is it an issue because I havent trained my model enough yet? For reference I am using the dlight trainer and typically use a VGG obstructed mask, although tests without a mask gave me worse results. Thanks

User avatar
bryanlyon
Site Admin
Posts: 433
Joined: Fri Jul 12, 2019 12:49 am
Answers: 33
Location: San Francisco
Has thanked: 3 times
Been thanked: 107 times
Contact:

Re: General Convert

Post by bryanlyon »

If you trained with 10 different faces you trained it incorrectly. Please go back to the training guide viewtopic.php?f=6&t=146 and follow it for ONE face per side. One face on A and one face on B. See also viewtopic.php?f=6&t=74#reuse .

User avatar
desertskye
Posts: 6
Joined: Thu Aug 06, 2020 2:15 am

Re: General Convert

Post by desertskye »

Ah okay, my bad. So should I continue with my model and only use one face, or would I be better off just making a new model? Thanks

User avatar
bryanlyon
Site Admin
Posts: 433
Joined: Fri Jul 12, 2019 12:49 am
Answers: 33
Location: San Francisco
Has thanked: 3 times
Been thanked: 107 times
Contact:

Re: General Convert

Post by bryanlyon »

I'd recommend deleting and starting over. But if you only used multiples on side B then you could delete the B decoder and start just that one over (the best way to do this is to start a new model then copy over the encoder and the A side decoder), but then you stand the chance of overtraining your A side. If you have enough data, it might be worth doing that.

User avatar
abigflea
Posts: 97
Joined: Sat Feb 22, 2020 10:59 pm
Answers: 1
Has thanked: 10 times
Been thanked: 20 times

Re: General Convert

Post by abigflea »

Why you need more sources.
I made some examples.

viewtopic.php?f=6&t=831
:o I dunno what I'm doing :shock:
RTX 2070 : GTX 1070 x2 : Ghetto 1060: Rx570

User avatar
desertskye
Posts: 6
Joined: Thu Aug 06, 2020 2:15 am

Re: General Convert

Post by desertskye »

Okay, thanks a lot. I'll probably just start over, i'm in no rush and would rather not have to deal with major problems or drops in quality. One last question, for the faces that i input into the training sessions, do I use (for example) the faces from one of obama's speeches or faces from multiple different obama speeches? Thanks

User avatar
torzdf
Posts: 853
Joined: Fri Jul 12, 2019 12:53 am
Answers: 122
Has thanked: 23 times
Been thanked: 170 times

Re: General Convert

Post by torzdf »

desertskye wrote:
Wed Aug 12, 2020 9:31 am
it seems that I'm swapping the face, but only the face from middle of forehead to chin. Is there a way to swap the whole image including background, where the new face is moving?
Generally, no. Faceswapping works well with faces because features in training sets tend to not change too much. With backgrounds/hair etc, it is a lot more features to change, meaning a lot more information to learn, which the model doesn't tend to have space for.

If it's only going halfway up the forehead, you will probably need to up the coverage a bit (there is a visual guide to coverage in the extract guide).
desertskye wrote:
Thu Aug 13, 2020 8:57 pm
do I use (for example) the faces from one of obama's speeches or faces from multiple different obama speeches? Thanks
More variety from more sources is better than 1 source.
My word is final

Post Reply