How to record a video for the sole purpose of training/face swapping?

Training your model
Forum rules
Read the FAQs and search the forum before posting a new topic.

Please mark any answers that fixed your problems so others can find the solutions.
Locked
User avatar
SmellyCheese
Posts: 11
Joined: Wed Jan 08, 2020 3:35 pm
Has thanked: 7 times

How to record a video for the sole purpose of training/face swapping?

Post by SmellyCheese » Thu Jan 09, 2020 3:58 am

Hey all, new to the forums. The future truly is now.

I saw in the guide, in training, your input B should have between 1,000 - 10,000 faces, ideally with varied expression.

Say I were to record a 60 FPS, 1080p video of a friend for face swaps. What would the best practice to obtain useable training faces be?

Sort of a niche question, but I'm going for indistinguishable from the original video level (once I figure out exactly what I'm doing)

Thanks guys!

User avatar
torzdf
Posts: 548
Joined: Fri Jul 12, 2019 12:53 am
Answers: 86
Has thanked: 16 times
Been thanked: 120 times

Re: How to record a video for the sole purpose of training/face swapping?

Post by torzdf » Thu Jan 09, 2020 2:47 pm

Shoot the video in various locations/various lighting situations/various poses. Maybe just have him interacting naturally with stuff, as you need as full a range of expressions and angles as possible.

When it comes to extracting, extract every N should be somewhere between 30-60 (i.e. every half a second to a second)
My word is final

User avatar
deephomage
Posts: 23
Joined: Fri Jul 12, 2019 6:09 pm
Has thanked: 2 times
Been thanked: 5 times

Re: How to record a video for the sole purpose of training/face swapping?

Post by deephomage » Thu Jan 09, 2020 4:51 pm

I've done deepfake work for clients who shot their own footage, and my suggestions would be:

Aim for about 15-20 minutes of high-quality video footage of each person's face to obtain enough images to make a face set for each person. Medium distance, front face views, as in an interview, work best.

The background of the face should be a neutral color or darker than the face to be swapped. Avoid bright light behind or above the face, even lighting is best. Fast motion of the head and motion blur is problematic.

As torzdf said, the video of each person's face should contain a variety of face angles, lighting settings, and expressions. Think of the model as trying to re-create a 3D image of the face, or of the head moving 180 degrees from shoulder to shoulder.

Profiles or side views of a face can't be easily deepfaked. Extreme close-ups of the face should be avoided, because of the model's face resolution limitations. The face is typically trained at 128x128 to 256x256 pixels and that is a tiny portion of the whole video frame.

User avatar
SmellyCheese
Posts: 11
Joined: Wed Jan 08, 2020 3:35 pm
Has thanked: 7 times

Re: How to record a video for the sole purpose of training/face swapping?

Post by SmellyCheese » Thu Jan 09, 2020 6:20 pm

Thank you guys!
The pursuit of knowledge and competence

User avatar
bryanlyon
Site Admin
Posts: 274
Joined: Fri Jul 12, 2019 12:49 am
Answers: 20
Location: San Francisco
Has thanked: 3 times
Been thanked: 78 times
Contact:

Re: How to record a video for the sole purpose of training/face swapping?

Post by bryanlyon » Thu Jan 09, 2020 6:23 pm

It's important to remember that there is a difference between Training data videos and the swap video. Making a quality training set needs a huge variety, but the swap video you want to minimize complexity in any way you can.

For training data, just run the camera, catch video when not actually filming the scenes. Everything you can get helps.

For swap, keep things simple, and straight forward.

User avatar
SmellyCheese
Posts: 11
Joined: Wed Jan 08, 2020 3:35 pm
Has thanked: 7 times

Re: How to record a video for the sole purpose of training/face swapping?

Post by SmellyCheese » Thu Jan 09, 2020 6:25 pm

Should I avoid flash?
The pursuit of knowledge and competence

User avatar
bryanlyon
Site Admin
Posts: 274
Joined: Fri Jul 12, 2019 12:49 am
Answers: 20
Location: San Francisco
Has thanked: 3 times
Been thanked: 78 times
Contact:

Re: How to record a video for the sole purpose of training/face swapping?

Post by bryanlyon » Thu Jan 09, 2020 6:27 pm

Flash, probably, but you should definitely use set lights. Lighting is probably the most important thing that people miss out on.

Locked