How can i merge converted face to the original frame?

Converting source frames into swapped frames
Forum rules
Read the FAQs and search the forum before posting a new topic.

Please mark any answers that fixed your problems so others can find the solutions.
Post Reply
User avatar
carolYL
Posts: 3
Joined: Fri Apr 10, 2020 5:51 pm

How can i merge converted face to the original frame?

Post by carolYL »

Hi there,
I used faceswap to extract face images for person A and person B from two videos. I trained my own GAN models using the face images extracted and was able to use the model to get person B's face based on person A's face. Now I need to merge the converted faces to the original video frames. Can i somehow use faceswap's utilities for that purpose?

Thanks!

User avatar
bryanlyon
Site Admin
Posts: 322
Joined: Fri Jul 12, 2019 12:49 am
Answers: 27
Location: San Francisco
Has thanked: 3 times
Been thanked: 87 times
Contact:

Re: How can i merge converted face to the original frame?

Post by bryanlyon »

Faceswap only works with the built in models for convert. The conversion tool doesn't ever actually use the extracted face images, those are only used for training. There is no way to merge external swaps using Faceswap. You'd have to significantly modify how Faceswap handles conversion to be able to recomposite those images back.

User avatar
carolYL
Posts: 3
Joined: Fri Apr 10, 2020 5:51 pm

Re: How can i merge converted face to the original frame?

Post by carolYL »

Thank you very much for the quick reply. Is there anyway i can take advantage of faceswap's converter using my own model(my model was written in PyTorch)?

User avatar
bryanlyon
Site Admin
Posts: 322
Joined: Fri Jul 12, 2019 12:49 am
Answers: 27
Location: San Francisco
Has thanked: 3 times
Been thanked: 87 times
Contact:

Re: How can i merge converted face to the original frame?

Post by bryanlyon »

You'd have to do some serious modifications to the convert process. If you have knowledge and experience with Python, you should be able to pull it off. Feel free to see how it works, start with scripts/convert and follow the process.

User avatar
carolYL
Posts: 3
Joined: Fri Apr 10, 2020 5:51 pm

Re: How can i merge converted face to the original frame?

Post by carolYL »

I have a question about _queue_out_frames function in class Predict(). This function seems to be putting the swapped faces back to the original frames, and it seems to be using item["swapped_faces"] = swapped_faces[pointer:pointer + num_faces] to do that, but i do not know where/what function does the actual merging. Could you please elaborate how the swapped faces are put back to the frame? (like how the alignment file is being used and which function is used to do the merging(where the color adjustment option, mask type option, scaling option are used))Thanks!

User avatar
torzdf
Posts: 638
Joined: Fri Jul 12, 2019 12:53 am
Answers: 92
Has thanked: 17 times
Been thanked: 128 times

Re: How can i merge converted face to the original frame?

Post by torzdf »

My word is final

Post Reply