Can you transfer to an inanimate object? (and one more)

[mrem]

Hi,
I apologize in advance if it is a party foul to ask two questions/requests in one post:

1) Is it possible to use faceswap to put a face on an inanimate object? (i.e., the end result being a magic mirror or talking statue)

2) I want to become a monthly supporter. Your link for patreon takes me to a page where I have to create an account. I already have a patreon account, but it doesn't allow me to proceed with the existing account.
2a) Additionally, is there a way I can give monthly without being listed?
M

[torzdf]

1) Is it possible to use faceswap to put a face on an inanimate object? (i.e., the end result being a magic mirror or talking statue)

It's possible, sure. Ultimately Faceswap is just learning what images are, it doesn't really understand what a face is. That said, I'm not sure how well the encoder will work if passing in very different types of images for the "a" and "b" sides. Also, you're going to have a hell of time generating alignments for inanimate objects, which you will need to do.

2) I want to become a monthly supporter. Your link for patreon takes me to a page where I have to create an account. I already have a patreon account, but it doesn't allow me to proceed with the existing account.
2a) Additionally, is there a way I can give monthly without being listed?

I'm not sure why that would be, and I'm also not sure about the public listing issue. You're probably best off contacting Patreon about that. Any support is obviously appreciated

You can probably set up monthly donations through paypal, but unfortunately we can't automate the perks that way, like we can with Patreon.

[bryanlyon]

Just wanted to add to what torzdf said and say that faceswap doesn't "animate" a static face. If that's what you're looking for Avatarify ( https://github.com/alievk/avatarify ) might be a better choice.

[mrem]

So, I could use Avatarify to create a "face" and dialogue (creating a base image by way of artwork or mannequin), and then could use Faceswap to move the "face" to create a deepfake of an historical figure speaking the dialogue? Or animate the artwork of a dead artist.
This gets my imagination flowing.
Would Faceswap be able to bring motion to a whole figure of a person, or just the face?

[torzdf]

Ahh., I think I may have misunderstood a bit.

Ultimately, Faceswap probably won't do what you want to do. It's for transferring expressions/movement etc from one face to another. So it can't create animations where they did not exist already in the source. So an inanimate object would still be inanimate, just with a different face.

[mrem]

Thank you for responding. I'm sorry that I'm not articulating clearly. I suspect that I changed horses midstream and lost you. ADD can do that.

Faceswap uses a target video (Training) and outputs to a destination video, if I understand proper.

I could create a video with Avatarify (that you recommended) and capture it with 3rd party software. That captured video could be used as the target video for use by Faceswap.

Faceswap would take that video and use it for training and then output it's video file. Example: I use Avatarify to create a face monologuing from a picture of a statue. I use a 3rd party software like OBS to capture the monologued video (let's call it MV#1). I then use MV#1 in Faceswap for training and have Faceswap put MV#1 on the video of said statue and output the results as MV#2, yes?

[bryanlyon]

No. Do not feed AI generated video into FS. It will undoubtedly fail. Just use Avatarify if that's the right one for you.

[mrem]

Ok.
What would you guys recommend software-wise for the task of turning a statue into a full body speaking actor?

[torzdf]

Probably compositing in After Effects, or the like, but others may have better suggestions.

[Bolos]

I'm also searching a way to make a target "speak" what i tell him. I had found first order motion, but the output is 256p wich is way too small for my wanted use. And i want to tanks for the link to avatarify !

I've seen the work of VFXChris Ume on YT (https://www.youtube.com/channel/UCGf4Ol ... lrgiH3jN3g) that seems to have a way around. I believe he uses a actor as the target and the final target as the model at first, then composite the face back to the orginial (the final target) with some neet vfx...

Otherwise there is this: http://grail.cs.washington.edu/projects/AudioToObama/
But it doesn't seems to be for public anytime soon, or ever...