How to Convert

Converting source frames into swapped frames
Forum rules
Read the FAQs and search the forum before posting a new topic.

Please mark any answers that fixed your problems so others can find the solutions.
Post Reply
User avatar
fixer2sonic
Posts: 4
Joined: Tue Jun 16, 2020 11:25 pm
Location: Seattle, Wa
Has thanked: 2 times

How to Convert

Post by fixer2sonic »

Hi! I'm a noob and feel lucky I have made it this far but hopefully someone can get me over the hump to the finish line. I have finished training my model to about 14000 iterations. Now I want to swap the faces in the final video by doing a convert. Please tell me what I've done wrong here and how to fix it if you'd all be so kind.

- input dir: to the original video I wish to swap my face onto.
- output dir: to a new folder I created for the final finished video to output to
- alignments: (I can't figure out if I point this to the alignment file created in my source video dir. after I cleaned it? Because I didn't know which one to use, I went ahead and tried that file.)
- reference video : I have seen posts to not put anything here and post to put something here so I'm not sure what to put
- model dir: I used Villain and pointed it to the "villain logs" dir. Is this right?

I then left most everything else alone except I changed mask type to "none" as I didn't use a mask.
And I changed the writer to Ffmpeg.

When I hit convert, It reads the alignments file I specified above, loads the ffmpeg plugin and then says "process exited." with a status at the bottom of the window that says; "Filed - convert.py. Return Code: 1"

Also, I just want to make sure I'm understanding something else correctly. I read that before I convert, I need to clean my alignments file. I already cleaned it in the previous step for extraction so am I supposed to do this again after training or is the one cleaned one I created before I trained the one I should use.

I have read all the FAQ's on this site, poured through the "convert' part of the forum but I can't find the answers I need to get the convert process to start. Any help is greatly appreciated.

Thanks!

User avatar
bryanlyon
Site Admin
Posts: 355
Joined: Fri Jul 12, 2019 12:49 am
Answers: 30
Location: San Francisco
Has thanked: 3 times
Been thanked: 93 times
Contact:

Re: How to Convert

Post by bryanlyon »

You need to use the alignments file that corresponds to the video you're converting. If you followed our recommendations during extract for training you probably used EEN to skip some faces, for convert you can't skip faces so you likely need to create a new alignment file with all the faces in it. If you did extract every face for training, then you already have the full alignments file and should use that one.

User avatar
fixer2sonic
Posts: 4
Joined: Tue Jun 16, 2020 11:25 pm
Location: Seattle, Wa
Has thanked: 2 times

Re: How to Convert

Post by fixer2sonic »

Thanks, bryanlyon, I appreciate the help! And I did follow your recommendations during extract for training and skipped some faces.

So, just so I understand what you're saying, once training both faces is done, I need to go back and do a new extract on both videos again (or would it be for just the video I wish to put a new face on) and this time I extract ALL faces, and after the faces are extracted, do I then go back through and remove all the unwanted faces (blurry, hand in front of face, other faces in scene I don't want, etc.) like I did the first time and then do a cleanup of the alignments file?

Sorry if that's a stupid question, I'm a noob and I am doing my best to understand why I'm doing what I'm doing in addition to just following a guide and doing it.

Thanks. :)
Time is the fire in which we burn.................

User avatar
bryanlyon
Site Admin
Posts: 355
Joined: Fri Jul 12, 2019 12:49 am
Answers: 30
Location: San Francisco
Has thanked: 3 times
Been thanked: 93 times
Contact:

Re: How to Convert

Post by bryanlyon »

Yes, this is covered in the Extract guide, but I know it can be confusing.

We generally recommend just doing one extract, but thinking ahead if it's going to be used for convert or just for training.

From the guide:
If you are extracting for convert, or you are extracting for convert AND will be using some of the faces for training, then leave this on 1 (i.e. extract from every frame)
Further, you can read about cleaning your alignments for convert here:
viewtopic.php?f=5&t=27#manual_conv

However unlike what you said, instead of removing blurry images, you want to keep them since you are converting here and want every frame to be covered.

User avatar
fixer2sonic
Posts: 4
Joined: Tue Jun 16, 2020 11:25 pm
Location: Seattle, Wa
Has thanked: 2 times

Re: How to Convert

Post by fixer2sonic »

Right on, thanks. Yeah, your extract guide is what I have been using, thank you.

To clarify, I have a video of a celebrity (Video *A*) that I want to put my face on. The video contains several other celebrities in it, too, that I don't want to put a face on. I only want to put my face on the one specific celebrity in that video.

I also have a video of myself (Video *B*) talking for about 20 mins. turning my head this way and that, different expressions, etc. that I used for training to be able to put on that celebrity in the other video.

So, if I'm understanding you correctly, I will run the extraction on both video *A* and video *B* and clean both alignment files and leave ALL faces extracted for both videos, right or when you said; "We generally recommend just doing one extract" you meant I only extract on Video *A* (destination video I want to put my face onto)? Then after the alignment file is cleaned, then I can do my convert?
Time is the fire in which we burn.................

User avatar
torzdf
Posts: 711
Joined: Fri Jul 12, 2019 12:53 am
Answers: 101
Has thanked: 19 times
Been thanked: 145 times

Re: How to Convert

Post by torzdf »

You just need to generate an alignments file for video A. You WILL need to clean this file.

This file tells the conversion process where, in the frame, the faces are to be swapped. If you don't clean the alignments file, then all faces in the video will be swapped.

Video B (i.e. the face you are swapping ONTO A) has no bearing in the convert process. All of this information is held within the model.
My word is final

User avatar
fixer2sonic
Posts: 4
Joined: Tue Jun 16, 2020 11:25 pm
Location: Seattle, Wa
Has thanked: 2 times

Re: How to Convert

Post by fixer2sonic »

Thanks a bunch, man, I really appreciate the help! :)
Time is the fire in which we burn.................

Post Reply