How to deepfake, but with input expression kept. (like snapchat)

Talk about AI and Deep Learning


Locked
User avatar
gserli
Posts: 1
Joined: Fri Feb 05, 2021 8:33 am

How to deepfake, but with input expression kept. (like snapchat)

Post by gserli »

Normally, deepfakes can do:
Input: One video of me saying something stupid.
Training data: Many video of celebrity X.
Output: Celebrity X saying something stupid. (with my voice)

But my goal is now:
Input A: One video of me saying something stupid.
Input B: One video of someone sitting on a cliff.
Output: Video of me saying something stupid and sitting on a cliff.

So it's face swap, without transforming facial expression and stuff. Just the input face appears normally in other's body.

The tool I need is more like the snapchat face swap filter. But is there anything more sophisticated?

User avatar
cosmico
Posts: 95
Joined: Sat Jan 18, 2020 6:32 pm
Has thanked: 13 times
Been thanked: 35 times

Re: How to deepfake, but with input expression kept. (like snapchat)

Post by cosmico »

Sounds like your asking for faceswap, but instead of Face B perfectly replicating Face A, you want to control Face B's face and voice while its on Face A's body. These tools definitely exist, like you mentioned Snapchats face swap filter, but I don't think any of these tools exists in a format as general purpose as faceswap. They tend to be super niche, or for specific purposes like snapchats face swap filter being for social media.

User avatar
bryanlyon
Site Admin
Posts: 793
Joined: Fri Jul 12, 2019 12:49 am
Answers: 44
Location: San Francisco
Has thanked: 4 times
Been thanked: 218 times
Contact:

Re: How to deepfake, but with input expression kept. (like snapchat)

Post by bryanlyon »

If it's for non-commercial uses, you can check out https://github.com/dunnousername/yanderifier which uses the First Order Motion model to transfer motion from one face to another. This doesn't match faceswap for quality, but provides a quick option.

Locked