[Guide] Extraction - A Workflow

Want to know about the Faceswap's Face Extraction process? Got tips, ideas or just want to learn about how it all works? Then this is the place for you


Forum rules

Read the FAQs and search the forum before posting a new topic.

This forum is for discussing tips and understanding the process involved for Extracting and preparing face sets for training a model in Faceswap.

If you have found a bug are having issues with the Extraction process not working, then you should post in the Extract Support forum.

Please mark any answers that fixed your problems so others can find the solutions.

User avatar
bryanlyon
Site Admin
Posts: 793
Joined: Fri Jul 12, 2019 12:49 am
Answers: 44
Location: San Francisco
Has thanked: 4 times
Been thanked: 218 times
Contact:

Re: [Guide] Extraction - A Workflow

Post by bryanlyon »

It's more art than science. Find the best images you can but don't go to in depth in your filtering, it's very easy to delete data that the ai might need to get a good swap.

I recommend removing only the wrong faces, misaligned faces, or really poor quality ones. You never know what the ai might need in order to learn.


Tags:
User avatar
dingdong420
Posts: 7
Joined: Sat Dec 21, 2019 3:43 am

No. json

Post by dingdong420 »

When I extract faces I don't seem to be creating an alignments.json file.

A couple things :

I did have "openfaceswap" installed previously. I uninstalled it before playing with "faceswap". Is this terminal? Like, do I need to reinstall windows terminal?

I meet exceed any hardware requirements, but Is there any software requisites we should be aware of before diving in? Like cuda versions and the like?

Thank you for taking the time to respond.

User avatar
bryanlyon
Site Admin
Posts: 793
Joined: Fri Jul 12, 2019 12:49 am
Answers: 44
Location: San Francisco
Has thanked: 4 times
Been thanked: 218 times
Contact:

Re: [Guide] Extraction - A Workflow

Post by bryanlyon »

Faceswap doesn't use Json anymore, and saves alignments in .fsa format. Please read the guide which mentions fsa files and how to use them.

User avatar
BLACKPINK
Posts: 13
Joined: Mon Dec 09, 2019 2:06 pm
Has thanked: 3 times

Re: [Guide] Extraction - A Workflow

Post by BLACKPINK »

I can’t hold shift to select all, it only allows me to select one only. And is it ok if I rename the alignment files? Now they got the same name inside the subfolders

User avatar
bryanlyon
Site Admin
Posts: 793
Joined: Fri Jul 12, 2019 12:49 am
Answers: 44
Location: San Francisco
Has thanked: 4 times
Been thanked: 218 times
Contact:

Re: [Guide] Extraction - A Workflow

Post by bryanlyon »

Yes you can rename files. If shift isn't working for you, try holding ctrl. Both work slightly differently but let you select now than one file. Worst case scenario is select one file at a time, copy the box into a new window, as you add each file then copy the whole collection back.

User avatar
Now
Posts: 10
Joined: Sat Jan 04, 2020 6:30 pm
Has thanked: 6 times
Been thanked: 1 time

Re: [Guide] Extraction - A Workflow

Post by Now »

can anyone help with "return code 1" when merging alignments

User avatar
torzdf
Posts: 2649
Joined: Fri Jul 12, 2019 12:53 am
Answers: 159
Has thanked: 128 times
Been thanked: 622 times

Re: [Guide] Extraction - A Workflow

Post by torzdf »

Does a crash report get generated?

Can you hit the "Generate" button and post the command?

My word is final

User avatar
SmellyCheese
Posts: 11
Joined: Wed Jan 08, 2020 3:35 pm
Has thanked: 7 times

Re: [Guide] Extraction - A Workflow

Post by SmellyCheese »

Is it okay to extract your input A and B with different masks? Is it detrimental?

The pursuit of knowledge and competence

User avatar
torzdf
Posts: 2649
Joined: Fri Jul 12, 2019 12:53 am
Answers: 159
Has thanked: 128 times
Been thanked: 622 times

Re: [Guide] Extraction - A Workflow

Post by torzdf »

Should be fine. You can also add additional masks with the mask tool

My word is final

User avatar
SmellyCheese
Posts: 11
Joined: Wed Jan 08, 2020 3:35 pm
Has thanked: 7 times

Re: [Guide] Extraction - A Workflow

Post by SmellyCheese »

You can use several masks for one faceset? Hm, have you noticed better results with that? And would you have to restart training?

Also, if you find the time, could you write out a brief workflow on how to use several masks. Thanks!

The pursuit of knowledge and competence

User avatar
sergor
Posts: 18
Joined: Wed Jan 15, 2020 7:59 pm
Has thanked: 9 times
Been thanked: 1 time

Re: Cleaning up

Post by sergor »

dingdong420 wrote: Sat Dec 21, 2019 5:56 am

I can not find directions on what makes a "clean" set of images. Could you eleborate on what should and shouldn't be deleted when cleaning up an extraction folder?

Hi there, I myself is one noob that would drive a moderator nuts - however first and foremost the first set of data (pictures) are of the outmost importance - with that said:

When extraction is complete -most often you don't know what the machine recognizes BUT let's say you have five photos of faces it will have the file name XX_0.JPG - in cases where the machine can't define the input (false positive) - it could be a warped frame, hand, butt (whatever) - and in my case 97% (out of 3000 images) it will have the following extension XX_1.jpg.

So to rough-cut your extraction - use command prompt in your extraction directory and use the "del *_1.jpg" - BUT DON'T FORGET to follow the guide for sorting when photos are deleted.

Hope that helps you out a bit.

User avatar
torzdf
Posts: 2649
Joined: Fri Jul 12, 2019 12:53 am
Answers: 159
Has thanked: 128 times
Been thanked: 622 times

Re: [Guide] Extraction - A Workflow

Post by torzdf »

SmellyCheese wrote: Sat Jan 18, 2020 4:46 am

You can use several masks for one faceset? Hm, have you noticed better results with that? And would you have to restart training?

Also, if you find the time, could you write out a brief workflow on how to use several masks. Thanks!

I mean you can "store" several masks per faceset, not that you can use them. You can only use the same mask for training A and B (at the moment).

I haven't had a chance to assess the newer masks yet.

My word is final

User avatar
torzdf
Posts: 2649
Joined: Fri Jul 12, 2019 12:53 am
Answers: 159
Has thanked: 128 times
Been thanked: 622 times

Re: Cleaning up

Post by torzdf »

sergor wrote: Sat Jan 18, 2020 10:05 am

So to rough-cut your extraction - use command prompt in your extraction directory and use the "del *_1.jpg" - BUT DON'T FORGET to follow the guide for sorting when photos are deleted.

Hope that helps you out a bit.

I don't recommend this. The order that detectors find faces is entirely arbitrary. Let's say it finds 2 faces in a frame, 1 of which is a false positive. The correct face could be in any of position of 0 and 1. By deleting all of the faces in position 1, you are giving yourself a 50/50 chance of either deleting a face or a false positive.

My word is final

User avatar
SmellyCheese
Posts: 11
Joined: Wed Jan 08, 2020 3:35 pm
Has thanked: 7 times

Re: [Guide] Extraction - A Workflow

Post by SmellyCheese »

I extracted my A with components and my B with vgg-obstructed and got poor results. Could that have been why? This is before I read you can only train with one mask

The pursuit of knowledge and competence

User avatar
9112097
Posts: 3
Joined: Sun Jan 19, 2020 11:02 pm

Re: [Guide] Extraction - A Workflow

Post by 9112097 »

不是很懂,能做一个视频教程吗?

[mod]Machine translation: Not very understanding, can you make a video tutorial?[/mod]

User avatar
omniplex
Posts: 1
Joined: Wed Jan 22, 2020 2:34 am

Re: [Guide] Extraction - A Workflow

Post by omniplex »

hey this looks like a lot of work to put all this together so first off thanks. second though ive read through this a few times and I thought I understood, and then I started having very basic questions that don't seem to be addressed in simple terms. during the extraction are we extracting the face that we want to use, or are we extracting the face we don't want to use, that we then replace later? I know its probably an obvious thing for most but the answer to that simple question would help me a lot! thanks!

User avatar
deephomage
Posts: 33
Joined: Fri Jul 12, 2019 6:09 pm
Answers: 1
Has thanked: 2 times
Been thanked: 8 times

Re: [Guide] Extraction - A Workflow

Post by deephomage »

To train a model, you need to extract both pairs of faces from video clips, to make a face set. For example, to swap Keanu Reeves and Nic Cage, you would extract Keanu and Nic faces from a variety of sources, to make a training set for each. If the video clips also contain unwanted faces, just delete the unwanted faces.

User avatar
korrupt78
Posts: 50
Joined: Wed Jan 29, 2020 1:34 am
Has thanked: 2 times
Been thanked: 1 time

Re: [Guide] Extraction - A Workflow

Post by korrupt78 »

I'm confused: What's the purpose of merging alignment files when merging training sets given that alignment files aren't actually used during training?

User avatar
deephomage
Posts: 33
Joined: Fri Jul 12, 2019 6:09 pm
Answers: 1
Has thanked: 2 times
Been thanked: 8 times

Re: [Guide] Extraction - A Workflow

Post by deephomage »

In a broad sense, a clean set of images means keeping the extracted faces you want to swap, deleting unwanted faces extracted from the video clip and also deleting any extracted objects falsely identified as faces.

User avatar
korrupt78
Posts: 50
Joined: Wed Jan 29, 2020 1:34 am
Has thanked: 2 times
Been thanked: 1 time

Re: [Guide] Extraction - A Workflow

Post by korrupt78 »

deephomage wrote: Wed Feb 12, 2020 8:07 pm

In a broad sense, a clean set of images means keeping the extracted faces you want to swap, deleting unwanted faces extracted from the video clip and also deleting any extracted objects falsely identified as faces.

Was that in response to me? Not trying to be rude, but genuinely confused because it doesn't seem to be answering the question I was asking at all. Maybe I did a bad job explaining. From the top:

  1. Sometimes I extract for conversion, in which case I extract every frame, and keep the alignments file, as the alignments file is what will be used for the conversion process.

  2. Sometimes I extract for training, in which case I'll only extract every 10-15 frames, and discard the alignments file, because the training phase only makes use of the images, not the alignment files.

  3. Sometimes I build a training set from images extracted from multiple videos, which means multiple extraction steps, followed by copy the contents of multiple image folders into one image folder.

The above Extract guide explains how to merge the alignment files from multiple extractions into a single alignment file, which only makes sense in the context of assembling a training set - you would never need to merge alignment files for a conversion, because conversions are done on single videos.

But if I'm not using the alignment file for training, then what need is there to merge them? Why bother merging alignment files into a single alignment file that you won't be using for anything?

It doesn't make sense, which to me is a red flag that I'm misunderstanding something potentially important.

Post Reply