A Short Hello with some questions

[K3l3vr4]

Hello, I'm new to this forum and find the topic very interesting. I have a computer with decent computing power here, but before I try extensively and let my PC work senselessly for weeks, I would like to inquire if you could help me here already, as some details are not entirely clear to me (if I missed the link for it, I'm sorry).
For example,

• if one has a video of oneself and wants to put their face onto an "original" where the person is wearing glasses, does the software recognize it? Or does it pixelate the object (the glasses), or even cause image distortions?

-if one has a movie and wants to have fun inserting oneself into that movie, is it sufficient to teach the "learning" face during training, and then the AI recognizes the other faces, leaving them unchanged and only altering the trained one?

• Would one need to "train" the corresponding face in every sequence, or is a partial section from, for example, 5 minutes of film enough to replace the face throughout the entire movie?

Since these are purely private fun projects, I don't necessarily want to let the PC work for weeks on projects only to delete gigabytes of data afterwards because everything is "garbage," if you understand.

I understand that I need to extract both Face A and Face B, that I need to train Model AB and Model BA (Face A --> Face B / Face B --> Face A), and then convert the old video with the corresponding model. I don't understand why I have to train 'back and forth,' but if that's how it is, then so be it. Therefore, it would be nice to get these questions answered, as I haven't become any wiser through searching the forum here.

I hope for your experience, and thanks for the help.

[bryanlyon]

Hello and Welcome.

• * if one has a video of oneself and wants to put their face onto an "original" where the person is wearing glasses, does the software recognize it? Or does it pixelate the object (the glasses), or even cause image distortions?

As long as a decent chunk of the training data has the glasses, you can actually sort of do either. Our masking model can leave the glasses if you wanted, or mask them out to be replaced by the swap.

• Would one need to "train" the corresponding face in every sequence, or is a partial section from, for example, 5 minutes of film enough to replace the face throughout the entire movie?

You'll actually want to use not just the one movie, but several other movies to get the best results. You will always get better results from more variety in your data, and one movie is unlikely to have the best variety. Please be sure to read the guides starting with the Extract guide viewtopic.php?t=27 which may be a lot to go through but is the best bet to get the best result.

[K3l3vr4]

If I understand you correctly, it comes down to this (fictional/theoretical): If I want to "smuggle" myself into an Indiana Jones like Harrison Ford, I shouldn't extract the entire film, but rather mainly passages with his face from all sorts of movies, so that Harrison's face becomes best known to the AI from all angles. Likewise, I need as much (preferably HD) video material of my face in as many angles as possible to extract the face and thus train both faces together. After a correspondingly detailed and 'lengthy' training phase (the introduction and instructions), could I then take a film with Harrison Ford and have his face replaced by mine, even if there are scenes in the film that the AI doesn't know, and even if other faces appear?

I'm just taking this as an example, because if one were to take a sequential snippet of Harrison Ford in DVD quality, from as many scenes as possible, string them together using a video tool to give the FaceSwap software a chance, for instance, to generate a model from 30 minutes of film (consisting only of Harrison Ford's face), and had one's own HD recorded material, then the question would be whether when converting the film with Harrison Ford using the generated model, the face can be replaced in every scene (because the AI recognizes it) or if it's only replaced in the specific scene where the recognition exists. If I understand the introduction and explanations and tutorials correctly, it should be possible, or am I mistaken somewhere?

Am I understanding this correctly?

Thank you for your help once again

[torzdf]

If I understand you correctly, it comes down to this (fictional/theoretical): If I want to "smuggle" myself into an Indiana Jones like Harrison Ford, I shouldn't extract the entire film, but rather mainly passages with his face from all sorts of movies, so that Harrison's face becomes best known to the AI from all angles. Likewise, I need as much (preferably HD) video material of my face in as many angles as possible to extract the face and thus train both faces together. After a correspondingly detailed and 'lengthy' training phase (the introduction and instructions), could I then take a film with Harrison Ford and have his face replaced by mine, even if there are scenes in the film that the AI doesn't know, and even if other faces appear?

Yes. You will also want a wide variety of lighting conditions + expressions. The more you give the AI the better it can generalize.

I'm just taking this as an example, because if one were to take a sequential snippet of Harrison Ford in DVD quality, from as many scenes as possible, string them together using a video tool to give the FaceSwap software a chance, for instance, to generate a model from 30 minutes of film (consisting only of Harrison Ford's face), and had one's own HD recorded material, then the question would be whether when converting the film with Harrison Ford using the generated model, the face can be replaced in every scene (because the AI recognizes it) or if it's only replaced in the specific scene where the recognition exists. If I understand the introduction and explanations and tutorials correctly, it should be possible, or am I mistaken somewhere?

I make you correct. It is important to remember, you are not swapping for specific scenes, you are swapping for specific identities.

I do recommend trying to keep your sources (on each side) within a similar threshold of age (for example, pick films of Ford within a 10 year range of the film you are aiming for). This will be less important if you are removing Ford's face, rather than swapping it in, but it is worth remembering.