[Guide] Extraction - A Workflow

Want to know about the Faceswap's Face Extraction process? Got tips, ideas or just want to learn about how it all works? Then this is the place for you


Forum rules

Read the FAQs and search the forum before posting a new topic.

This forum is for discussing tips and understanding the process involved for Extracting and preparing face sets for training a model in Faceswap.

If you have found a bug are having issues with the Extraction process not working, then you should post in the Extract Support forum.

Please mark any answers that fixed your problems so others can find the solutions.

User avatar
torzdf
Posts: 2649
Joined: Fri Jul 12, 2019 12:53 am
Answers: 159
Has thanked: 128 times
Been thanked: 623 times

Re: [Guide] Extraction - A Workflow

Post by torzdf »

korrupt78 wrote: Wed Feb 12, 2020 6:59 pm

I'm confused: What's the purpose of merging alignment files when merging training sets given that alignment files aren't actually used during training?

If you're training with a mask or using Warp to Landmarks, then you need an alignments file, as stated here:
viewtopic.php?f=5&t=27#why

My word is final


Tags:
User avatar
korrupt78
Posts: 50
Joined: Wed Jan 29, 2020 1:34 am
Has thanked: 2 times
Been thanked: 1 time

Re: [Guide] Extraction - A Workflow

Post by korrupt78 »

torzdf wrote: Wed Feb 12, 2020 10:03 pm

If you're training with a mask or using Warp to Landmarks, then you need an alignments file, as stated here:
viewtopic.php?f=5&t=27#why

Great, that helps. I'm just getting into learning the Manual tool now, and made a decision to figure out masks and landmarks after I get the basics of the Manual tool, so I haven't actually looked into them yet. I'll postpone further comments on this question until I do.

User avatar
230V
Posts: 20
Joined: Sun Jan 19, 2020 6:20 pm
Answers: 1
Has thanked: 7 times
Been thanked: 2 times

Re: [Guide] Extraction - A Workflow

Post by 230V »

thx for the guide!

but how to resize "Frame" windows so i can see the entire picture?
footage is 4k, display not..

best regards

User avatar
torzdf
Posts: 2649
Joined: Fri Jul 12, 2019 12:53 am
Answers: 159
Has thanked: 128 times
Been thanked: 623 times

Re: [Guide] Extraction - A Workflow

Post by torzdf »

My word is final

User avatar
Kevin_Z
Posts: 11
Joined: Tue Apr 28, 2020 2:36 am
Has thanked: 1 time
Been thanked: 1 time

Re: [Guide] Extraction - A Workflow

Post by Kevin_Z »

The extraction procedure is working on CPU or GPU? It is kind of slow, when I ran it on AMD 3950x, and CPU was all FULL.

User avatar
torzdf
Posts: 2649
Joined: Fri Jul 12, 2019 12:53 am
Answers: 159
Has thanked: 128 times
Been thanked: 623 times

Re: [Guide] Extraction - A Workflow

Post by torzdf »

It uses both, fairly extensively

My word is final

User avatar
AaronLaw
Posts: 6
Joined: Mon May 11, 2020 10:42 am
Has thanked: 6 times

Re: [Guide] Extraction - A Workflow

Post by AaronLaw »

Dear facewap:

Thanks to facewap for providing such a magical application, when I learned to extract according to the guide, I found that there are some abnormal situations, I don't know if other friends have encountered:

  1. Extract the face in the video, the face in several frames of the picture is rotated at an angle, which will cause the face position to be misaligned in the subsequent training;

  2. Among the extracted results, there are still pictures that are not faces. Can I directly delete them manually, and then use the “Clean the Alignments File” in the guide to process?

Image

Image

I would like to consult how to solve this situation.

       Thanks ~ ···

User avatar
torzdf
Posts: 2649
Joined: Fri Jul 12, 2019 12:53 am
Answers: 159
Has thanked: 128 times
Been thanked: 623 times

Re: [Guide] Extraction - A Workflow

Post by torzdf »

AaronLaw wrote: Mon May 11, 2020 11:44 am
  1. Among the extracted results, there are still pictures that are not faces. Can I directly delete them manually, and then use the “Clean the Alignments File” in the guide to process?

Yes.

The slightly misaligned faces can be fixed with the manual alignments tool.

My word is final

User avatar
meDaniel
Posts: 2
Joined: Thu Aug 06, 2020 6:58 am

Re: [Guide] Extraction - A Workflow

Post by meDaniel »

deephomage wrote: Thu Jan 23, 2020 6:47 pm

To train a model, you need to extract both pairs of faces from video clips, to make a face set. For example, to swap Keanu Reeves and Nic Cage, you would extract Keanu and Nic faces from a variety of sources, to make a training set for each. If the video clips also contain unwanted faces, just delete the unwanted faces.

HI, regarding this, I have a few source videos of myself.

As you mentioned, I should extract my faces from these source; but do I output them to the same folder?
Example:

Code: Select all

/Output Pic/Myself

And same for whoever I want to swap my face with?
Example:

Code: Select all

/Output Pic/Whoever

Just verifying and trying to understand whether I am on the right path.

User avatar
bryanlyon
Site Admin
Posts: 793
Joined: Fri Jul 12, 2019 12:49 am
Answers: 44
Location: San Francisco
Has thanked: 4 times
Been thanked: 218 times
Contact:

Re: [Guide] Extraction - A Workflow

Post by bryanlyon »

[mention]meDaniel[/mention] Yes, you can do that. You'll just need to follow the merging instructions in the guide to merge your alignments files after you finish extracting. Personally, I extract to different folders, then merge, but that's just a personal choice.

User avatar
torzdf
Posts: 2649
Joined: Fri Jul 12, 2019 12:53 am
Answers: 159
Has thanked: 128 times
Been thanked: 623 times

Re: [Guide] Extraction - A Workflow

Post by torzdf »

You definitely should be outputting to different folders, and then looking to fix up the alignments for each extract run prior to merging.

Once the files are merged, you won't be able to use the Manual tool to fix up any issues.

My word is final

User avatar
meDaniel
Posts: 2
Joined: Thu Aug 06, 2020 6:58 am

Re: [Guide] Extraction - A Workflow

Post by meDaniel »

Thanks for both the tips.

I will redo my extraction again. Because I extracted my own face from two source into the same folder.

Will try the method with the least issue =D

Guess I will be staring in some movies soon.

User avatar
FuorissimoX
Posts: 56
Joined: Mon Sep 21, 2020 6:49 am
Location: Italy
Has thanked: 10 times
Been thanked: 2 times

Re: [Guide] Extraction - A Workflow

Post by FuorissimoX »

Hi, I'm trying the "Manually Fixing" function but I'm having trouble understanding what is right and what is wrong. Can you post photos? It would be easier. Thank you

User avatar
DildoSaggins
Posts: 1
Joined: Mon Nov 09, 2020 10:47 am

Re: [Guide] Extraction - A Workflow

Post by DildoSaggins »

I'm failing to understand exactly what I should have/keep from the extraction process to be used in either training or conversion. For example, following the extraction guide, for my "B" face(the face to be inserted INTO the video), when I finished, I now have two folders.

"B_faces_source" - which contains the original unaltered images to get faces from, as well as the alignments file I ended up with after following the guide.

and

"B_faces_extracted" - which contains the cropped/rotated face images generated during the extraction process and NO alignments files.

Once I get to the training section, it specifies that I should be feeding in the face files from the extracted folder, and not the source folder. But what about the alignment file? Can I use the alignment file from the source folder with the images from the extracted folder? Or do I feed the trainer both the original uncropped images from the source folder and that alignment file, and not actually use those cropped images in the extracted folder at all? When I did my extracting I used a mask (vgg-clean), so I know I definitely need to train with an alignments file.

Last edited by DildoSaggins on Mon Nov 09, 2020 12:52 pm, edited 1 time in total.
User avatar
torzdf
Posts: 2649
Joined: Fri Jul 12, 2019 12:53 am
Answers: 159
Has thanked: 128 times
Been thanked: 623 times

Re: [Guide] Extraction - A Workflow

Post by torzdf »

Yes, you need the extracted faces of B and a copy of the alignments file that was generated and currently resides with the source B images.

My word is final

User avatar
Joe100
Posts: 5
Joined: Fri Nov 13, 2020 11:30 am
Has thanked: 8 times

Re: [Guide] Extraction - A Workflow

Post by Joe100 »

Hi all

First, again a BIG thank you for all the Faceswap team's hard work designing and supporting this massive project. We should all be supporting the team by signing up to Patreon.

Second, can I ask a question about alignments for conversion please?
I have an alignment file for training where I painstakingly adjusted each alignment mapped to the face. This extract used an EEN of every 10 frames or so.
I have now extracted a full alignment file for conversion with EEN 1. Is there any way to import the training alignments which I worked so hard on? I could then use those corrected alignments as a basis to correct the other 90% of "conversion alignments"?
Thanks very much for advice

User avatar
torzdf
Posts: 2649
Joined: Fri Jul 12, 2019 12:53 am
Answers: 159
Has thanked: 128 times
Been thanked: 623 times

Re: [Guide] Extraction - A Workflow

Post by torzdf »

Sadly there isn't an "out of the box" way, no.

If you were comfortable with Python scripting, then this could be done, yes.

My word is final

User avatar
manintan
Posts: 4
Joined: Sat Nov 21, 2020 10:41 pm

Re: [Guide] Extraction - A Workflow

Post by manintan »

bryanlyon wrote: Mon Dec 16, 2019 3:49 pm

No, this is really not how the alignments are used. Even when copying from a previous frame, we recommend just using that as the starting point of detecting so that you get valid alignments for the face, otherwise you are just adding bad data that will confuse the model.

There are times when the mask is not on the face because the person is moving and wasnt properly detected. It would be helpful if moving the initial mask points in the manual frame would flow and carry over to the rest of the thumbs in manual, or a way to just have one mask in a frame say be copied to a specific number of other frames. Im talking about those points we can move around.

User avatar
Joe100
Posts: 5
Joined: Fri Nov 13, 2020 11:30 am
Has thanked: 8 times

Re: [Guide] Extraction - A Workflow

Post by Joe100 »

Thank you Torzdf.
Yeah I'm not a clever programmer with python skills, sadly!

User avatar
manintan
Posts: 4
Joined: Sat Nov 21, 2020 10:41 pm

Re: [Guide] Extraction - A Workflow

Post by manintan »

Deleted my alignments file but still have the extracted faces. How do I make a new alignments file?

Post Reply