Page 1 of 1

How to deepfake, but with input expression kept. (like snapchat)

Posted: Fri Feb 05, 2021 8:48 am
by gserli

Normally, deepfakes can do:
Input: One video of me saying something stupid.
Training data: Many video of celebrity X.
Output: Celebrity X saying something stupid. (with my voice)

But my goal is now:
Input A: One video of me saying something stupid.
Input B: One video of someone sitting on a cliff.
Output: Video of me saying something stupid and sitting on a cliff.

So it's face swap, without transforming facial expression and stuff. Just the input face appears normally in other's body.

The tool I need is more like the snapchat face swap filter. But is there anything more sophisticated?


Re: How to deepfake, but with input expression kept. (like snapchat)

Posted: Thu Feb 18, 2021 4:07 pm
by cosmico

Sounds like your asking for faceswap, but instead of Face B perfectly replicating Face A, you want to control Face B's face and voice while its on Face A's body. These tools definitely exist, like you mentioned Snapchats face swap filter, but I don't think any of these tools exists in a format as general purpose as faceswap. They tend to be super niche, or for specific purposes like snapchats face swap filter being for social media.


Re: How to deepfake, but with input expression kept. (like snapchat)

Posted: Thu Feb 18, 2021 9:39 pm
by bryanlyon

If it's for non-commercial uses, you can check out https://github.com/dunnousername/yanderifier which uses the First Order Motion model to transfer motion from one face to another. This doesn't match faceswap for quality, but provides a quick option.