[Guide] Extraction - A Workflow

Extracting faces for training and converting
Forum rules
Read the FAQs and search the forum before posting a new topic.

Please mark any answers that fixed your problems so others can find the solutions.
User avatar
torzdf
Posts: 1008
Joined: Fri Jul 12, 2019 12:53 am
Answers: 128
Has thanked: 28 times
Been thanked: 193 times

[Guide] Extraction - A Workflow

Post by torzdf »

NB: This guide was correct at time of writing, but things do change. I will try to keep it updated.

Contents



Introduction


A lot of people get overwhelmed when they start out Faceswapping, and many mistakes are made. Mistakes are good. It's how we learn, but it can sometimes help to have a bit of an understanding of the processes involved before diving in.

In this post I will detail a workflow for Extraction. I'm not saying that this is the best workflow, or what you should do, but it works for me, and will hopefully give you a good jump off point for creating your own workflow.

I will be using the GUI for this guide, but the premise is exactly the same for the cli (all the options present in the GUI are availablle in the cli).

Why Extract?


At the highest level, extraction consists of three phases: detection, alignment and mask generation. There are several plugins for each of these. Their pros and cons are detailed in the tooltips (for the GUI) or the help text (for the cli), so I won't be covering specifics. However, detection is the process of finding faces within a frame, alignment is finding the "landmarks" within a face and orienting the faces consistently. Finally mask generation is creating a "mask", that is identifying which parts of the final face image are face and which are background/obstructions.

Extracting serves 2 main purposes:

  1. To generate a set of faces, and optionally an alignments file and mask, for training the model

  2. To generate an alignments file and mask for converting your final frames

There is technically a 3rd purpose which is when you are extracting for converting and want some of these faces for training as well. I will also cover this off.

Whilst the extracted faces are not required for convert (only the alignments file is), it is useful to have them so that we can clean up our dataset for the conversion process.

The Alignments File
The alignments file holds information about all the faces it finds in each frame, specifically where the face is, and where the 68 point landmarks are:
Image
It also stores any masks that have been extracted for each face.

The purpose of the alignments file is three-fold:

  1. Training: It provides the face landmarks if "Warp to Landmarks" is enabled.

  2. Converting: It tells the convert process where the face is in the original frame to be replaced.

  3. Training and converting: It stores any masks that may be used for either of these processes.

So now we know why we need to extract faces, how do we nail down a good workflow?

Extracting


The first step, whatever your reason for creating a faceset is to actually extract the faces from the frames.

  • Data
    This is where we select our input videos or images, and our output location.

    Image 1.jpg
    Image 1.jpg (95.14 KiB) Viewed 151309 times
    • Input Dir: First up we need a source. This can be set to either a video file (video icon) or to a folder containing a series of images (picture icon).

    • Output Dir: Next we need to tell the process where to save the extracted faces to. Click the folder icon next to select an output location.

    • I ignore the next few options, so you probably should too, but the explanation of each of them is below:

      • Alignments: This option is in case you want to specify a different name/location for the alignments file. It's a lot easier to leave this blank, and let the process save it out to the default location (which will be in the same location as the source frames/video).

  • Plugins
    The plugins are used for first detecting faces in an image, reading the faces landmarks to align the image and creating a mask using a variety of masking methods.

    plugins00.jpg
    plugins00.jpg (48.05 KiB) Viewed 156299 times
    • Detector, Aligner: Select the ones you want to use. At the time of writing the best detector is S3FD and the best aligner is FAN

    • Masker: Select which masker(s) you would like to use. The landmark based masks (Components and Extended) are automatically generated during extraction, so you cannot select these, however they are detailed below so that you know how they perform. Any masker selected here will generate a mask in addition to the landmark based masks. Multiple masks can be generated (by selecting more than 1 masker), however be aware that all of these maskers use the GPU, so the more maskers you add, the slower the extract speed will be.

      Each masker has different pros and cons. If you wish to use a Neural Network based mask for training or converting your frames then you should choose a mask here.

      NB: Masks are a new addition, so information about the best to use is fairly thin on the ground. Whilst you can only extract one mask during the extraction process, you can extract more masks/add masks to existing alignments files, as well as generate previews, with the Mask Tool.

      • None - Don't create a mask.

      • Components - A mask generated from the 68 point landmarks found in the alignments phase. This is a multi-part mask built around the outside points of the face.

        mask_components.png
        mask_components.png (122.11 KiB) Viewed 156294 times
      • Extended - One of the main problems with the components mask is that it only goes to the top of the eyebrows. The Extended Mask is based off the components mask, but attempts to go further up the forehead to avoid the "double eyebrow" issue, (i.e. when both the source and destination eyebrows appear in the final swap.).

        mask_extended.png
        mask_extended.png (126.5 KiB) Viewed 156294 times
      • Unet-DFL - A Neural Network mask designed to provide smart segmentation of mostly frontal faces. The mask model has been trained by community members.

        mask_unet-dfl.png
        mask_unet-dfl.png (118.75 KiB) Viewed 156294 times
      • VGG-Clear - A Neural Network mask designed to provide smart segmentation of mostly frontal faces clear of obstructions.

        mask_vgg-clear.png
        mask_vgg-clear.png (137.98 KiB) Viewed 156294 times
      • VGG-Obstructed - designed to provide smart segmentation of mostly frontal faces. The mask model has been specifically trained to recognize some facial obstructions (hands and eyeglasses).

        mask_vgg-obstructed.png
        mask_vgg-obstructed.png (130.46 KiB) Viewed 156294 times
    • Normalization: This processes the image fed into the extractor to better find faces in difficult lighting conditions. Different normalization methods work with different datasets, but I find "hist" to be the best all-rounder. This will slightly slow extraction, but it can lead to better alignments:

    • The next few options can also be ignored. Again, I will detail what they do so you can make your own call:

      • Rotate Images: This is entirely unnecessary with all current detectors except for maybe CV2-DNN. The current detectors are perfectly capable of finding faces at any orientation, so this will just slow down your process for little or no gain.

  • Face Processing
    Options for face filtering and selection are defined here.

    Image 4.jpg
    Image 4.jpg (22.08 KiB) Viewed 147146 times
    • Min-size: I normally set this at a fairly low value, but above zero, to filter out anything that is clearly too small to be a face. It will, however, depend on the size of your input frames, and then the size of the faces within your frames:

    • I ignore the next few options, for the reasons stated below, but you can make your own call. An explanation of each of them is below:

      • Filter/nFilter, Ref Threshold: These options are for filtering in/out unwanted faces. I don't use it. I prefer to do this myself at the next stage, as I will always be more effective at filtering out faces than a machine is, and it doesn't take long. Filtering faces also severely slows down the extraction process.

  • Output
    Options for the output processing of faces.

    • Extract Every N: This is the first option that will change depending on the purpose of this extraction.

      • If you are extracting for convert, or you are extracting for convert AND will be using some of the faces for training, then leave this on 1 (i.e. extract from every frame)

      • If you are extracting for training only, then set a value that seems sane. This will depend on the number of frames per second your input is. However, for a 25fps video, sane values are between about 12 - 25 (i.e. every half second to a second). Anything less than that and you will probably end up with too many similar faces. It is worth bearing in mind how many sources you are extracting from for your training set, and how many faces you want to have in your final training set, as well as how long your source is. These will all vary on a case by case basis.

    • Again, these next few options I generally skip, however explanations below:

      • Size: This is the size of the image holding the extracted face. Currently no models support above 256px so leave this at default

      • Save interval: I don't bother with, but it's up to you.

      • Debug Landmarks: Draws landmarks on the face. Only useful for debugging purposes.

  • Run
    We're now ready to review our options and run Extraction.

    • You should end up with a screen similar to below:

      Image 7.jpg
      Image 7.jpg (217.78 KiB) Viewed 147146 times
    • Hit Extract and wait for your faces to output and your alignments file to be generated.

Sorting


Now that we have extracted our faces, we need to clean up the dataset and the alignments file. The Extractor does a good job of getting faces, but it isn't perfect. It will have got some false positives, it will have failed to align some faces and it will have also extracted people who we don't want to swap. Most likely if you go to your faces folder you will have large sections of the output which look something like this:

Image-11.jpg
Image-11.jpg (24.87 KiB) Viewed 171489 times

Cleaning that up doesn't look like it will be fun! Fortunately we can make this easier. The quickest and easiest way to clean our dataset is to sort the faces into a meaningful order and then delete all the faces we don't want. The best sort method, by some distance, is "sort by face".

NB: Sorting by face is RAM intensive. It has to do a LOT of calculations. I have tested successfully sorting 22k faces on 8GB of RAM. If you have to sort more faces than can be held in RAM, then the process will automatically switch to a much slower method, so if you are RAM limited, then you may want to split your dataset into smaller subsets. Theoretically, the amount of RAM required is (n2 * 24) / 1.8 where n is the number of images. You will need to take any other RAM overhead in to account too (i.e. other programs running, the images loaded to RAM), but theoretically it means 30,000 images will take (300002 * 24) / 1.8 = 12,000,000,000 bytes or about 11GB.

Head over to the "Tools" tab in the GUI and then to the "Sort" sub-tab:

sort_main.jpg
sort_main.jpg (60.92 KiB) Viewed 77606 times
  • Data
    This is where we select the Faceset that we extracted in the previous step for sorting.

    • Input: Enter the folder which contains your extracted faces from the previous step.

    • Output: This should be empty. We want the faces to be sorted in place.

    sort_data.jpg
    sort_data.jpg (17.21 KiB) Viewed 77606 times
  • Sort Settings
    How we want to sort the faces and any other specific settings.

    • Sort By: Set this to "Face"

    • Backend: Optionally set this to GPU. Only do this if you have a GPU and you are not currently using your GPU for any other Machine Learning tasks (e.g. Training). It will make sorting go much faster.

    • You can ignore any other options in this section

    sort_by.jpg
    sort_by.jpg (57.18 KiB) Viewed 77601 times
  • Output
    How we want our sorted faces to be output.

    • Final Process: Make sure this is set to "Rename"

    • Keep: Make sure this is unchecked.

    • You can leave all of the other options untouched as they don't have any effect on rename tasks.

    sort_output.jpg
    sort_output.jpg (28.89 KiB) Viewed 77606 times
  • Run
    Leave all other options at default. We're now ready to review our options and run Sort.

    • You should end up with a screen similar to below:

      sort_settings.jpg
      sort_settings.jpg (193.65 KiB) Viewed 77606 times
    • Hit Sort to start the sorting process.

The process will start reading through the faces building up an identity for each. It will then cluster the faces together by similarity. The actual clustering process can take a long time as it has to compute a lot of data. Unfortunately there is no visual feedback as to it's progress, so please be patient.

Once complete you should find 99% of the faces sorted together:

Image-24.jpg
Image-24.jpg (24.68 KiB) Viewed 141601 times

And all the junk sorted together as well:

Image-25.jpg
Image-25.jpg (24.9 KiB) Viewed 141601 times

Now all you have to do is scroll through the faces folder and delete those faces out that you do not wish to keep.

Clean the Alignments File


Now we have removed all of the faces we don't want and just have a set that we do, we need to clean the alignments file. Why? Because all the information about the unwanted faces is still in the file, and this is likely to cause us problems in future. Cleaning the alignments file with the integrated tool also has the added advantage of renaming our faces back to their original filename, so it's a win-win.

Navigate to the "Tools" tab then the "Alignments" sub-tab:

alignments_main.jpg
alignments_main.jpg (72.3 KiB) Viewed 77596 times
  • Job
    This is a list of all of the different alignments jobs that are available in the tool.

    • Select "remove-faces":

    align2.jpg
    align2.jpg (47.64 KiB) Viewed 164613 times
  • Data
    The location of the resources we want to process.

    • Alignments File: select the alignments file that was generated during the extraction process It will be located in the same folder as the video you extracted from, or within the folder of the images you extracted from.

    • Faces Folder: select the faces folder that you outputted faces into as part of the extraction process (the same folder as you used for sort).

    • Leave the other options blank in this section as they are not required.

    alignments_data.jpg
    alignments_data.jpg (30.46 KiB) Viewed 77596 times
  • Run
    Leave all other options at default. We're now ready to review our options and clean the alignments file

    • You should end up with a screen similar to below:

      alignments_settings.jpg
      alignments_settings.jpg (218.65 KiB) Viewed 77596 times
    • Hit Alignments and let the process complete.

Once done your faces will have been renamed back to their default names, and all of the unwanted faces will have been removed from your alignments file.

The process will have backed up your old alignments file and put it next to your newly created file in it's original location. It will have the same name as your cleansed alignments file but with "backup_<timestamp>" appended on to the end of it. If you are happy that your new alignments file is correct, you can safely remove this backup file.

At this point, if you are extracting for convert (or the set is going to be used for convert and training) then you can entirely delete your faces folder. None of these faces are required any more. If you ever need to re-generate your face set, then this can be done with the Alignments Tool (extract job).

Manually Fix Alignments


Ok, we've extracted our faces, we've cleaned out all the trash, surely we're done now? Well hold on there fella. Sure, you could move on now, but do you want an ok swap or do you want a great swap?

Manual is useful for the following tasks:

  • Removing any left over "multiple faces" in a frame

  • Adding any alignments to those frames that are missing them

  • Fixing misaligned frames

  • Manually editing any masks you wish to use

Depending on what we are extracting our dataset for will dictate what we want to do here. If we are purely extracting for a training set, we could feasibly skip this step altogether, although it is a good idea to review the existing alignments to make sure any masks get built correctly.

If extracting for convert, then at the absolute minimum we want to fix any multiple faces in frames, and any frames missing alignments. This alone will improve the final swap. Depending on how thorough you want to be, you can then fix any poor alignments

I won't go into full details on how to use the Manual Tool. That is a guide in itself, but should be quite intuitive, with tooltips for all of the available options.

  • Launch the Manual Tool
    To launch the Manual tool first navigate to the "Tools" tab then the "Manual" sub-tab:

    manual_main.jpg
    manual_main.jpg (68.12 KiB) Viewed 77591 times
    • Data
      This is where we specify the alignments file and video file/frames folder we wish to work with.

      • Alignments: select the alignments file that was generated during the extraction process It will be located in the same folder as the video you extracted from, or within the folder of the images you extracted from.

      • Frames: select the video or the frames folder that you used as an input for the extraction process

      manual_data.jpg
      manual_data.jpg (25.1 KiB) Viewed 77591 times
    • Options
      This are optional actions that can be run. You won't generally need to change these, but I'll list what they do in case you do need to utilize them.

      • Regen Thumbs: The manual tool stores a cache of thumbnails for populating the Faces Viewer part of the tool. If you have used the old manual tool with your alignments file, then it is possible that the thumbs may have gone out of sync with the actual extracted faces. This option regenerates the thumbnails so that they are correct with the data that exists within the alignments file.

      • Single Process: In order to speed up the generation of the thumbnail cache, the process launches several threads to read the images from the video file. On some videos this causes the process to hang. If this happens to you, then select this option to extract the thumbnail in a single thread.

      manual_options.jpg
      manual_options.jpg (9.8 KiB) Viewed 77591 times
    • Run
      Leave all other options at default. We're now ready to review our options and launch the manual tool.

      • You should end up with a screen similar to below:

        manual_settings.jpg
        manual_settings.jpg (137.88 KiB) Viewed 77591 times
      • Hit the Manual button to launch the manual tool.

      • NB: It may take a minute or two for the Manual Tool to launch, as it relies on your GPU, so needs to initialize the aligners/maskers that it will use.

  • Manually Fixing for Converting
    For fixing for convert we want to do the following:

    • Multi-Faces

      • Change the filter mode to "Multiple Faces" in the filter pull down located between the Frames and Faces Viewer

      • You can right click faces in the Faces Viewer to delete faces you wish to delete

      • Alternatively, you can change the Editor to "Bounding Box" or "Extract Box" on the left-hand button bar and right click faces you want to delete from the Frame Viewer.

      • Use the transport buttons or "Z" and "X" to navigate to the previous or next frame with multiple faces.

      • Press "Ctrl+S" to save out your amended alignments file, or press the save icon located between the Frames and Faces Viewer.

    • Frames without faces

      • Change the filter mode to "No Faces" in the filter pull down located between the Frames and Faces Viewer

      • Change the edit mode to "Bounding Box" editor using the left=hand button bar

      • Use the transport buttons or "Z" and "X" to navigate to the previous or next frame without faces.

      • If you hit a frame where no face was detected but should have been, you can either click on the face and adjust the bounding box with the mouse, or you can copy the previous/next frame's alignments using the copy buttons on the left-hand side or using the "c" or "v" keys and edit those.

      • Repeat until your reach the last frame

      • Press "Ctrl+S" to save out your amended alignments file, or press the save icon located between the Frames and Faces Viewer.

    • Fixing alignments

      • Change the filter mode to "Has Faces" in the filter pull down located between the Frames and Faces Viewer

      • You can navigate through the frames with the transport controls, or click on a face in the Faces Viewer to jump straight to any frames that need attention.

      • If the alignments look incorrect, use the appropriate editor from the left-hand button bar to fix them up

      • Repeat until you reach the last frame

      • Press "Ctrl+S" to save out your amended alignments file, or press the save icon located between the Frames and Faces Viewer.

    A backup of your alignments file will have been created each time you saved during manual adjustment. If you are happy that your new alignments file is correct, you can delete these backups. They will be located in the same folder as the original alignments file.

  • Manually Fixing for a Training Set
    For fixing for training we want to do the following:

    • Fixing alignments

      • Change the filter mode to "Has Faces" in the filter pull down located between the Frames and Faces Viewer

      • You can navigate through the frames with the transport controls, or click on a face in the Faces Viewer to jump straight to any frames that need attention.

      • If the alignments look incorrect, use the appropriate editor from the left-hand button bar to fix them up

      • Repeat until you reach the last frame

      • Press "Ctrl+S" to save out your amended alignments file, or press the save icon located between the Frames and Faces Viewer.

    A backup of your alignments file will have been created each time you saved during manual adjustment. If you are happy that your new alignments file is correct, you can delete these backups. They will be located in the same folder as the original alignments file.

Once the alignments are fixed, you will need to regenerate your faceset. This is because the face image will have changed due to moving the landmarks, so the training process will not be able to find the faces you amended.

Firstly delete your output faces folder. You don't want this around any more.

You can then hit the Folder button located next to the save icon to extract faces straight from the alignments file.

An alternative method is listed below, but if you have successfully extracted from the Manual Tool, you can jump straight to Extracting a training set from an alignments file

  • Next navigate to the "Tools" tab then the "Alignments" sub-tab:

    align1.jpg
    align1.jpg (52.65 KiB) Viewed 164613 times
    • Job
      This is a list of all of the different alignments jobs that are available in the tool.

      • Select "extract":

        align9.jpg
        align9.jpg (47.2 KiB) Viewed 164603 times
    • Data
      The location of the resources we want to process.

      • Alignments File: select the alignments file that was generated when after cleaning in the last step. It will be located in the same folder as the video you extracted from, or within the folder of the images you extracted from.

      • Faces Folder: select an empty folder where you want the outputted faces to go.

      • Frames Folder: select the video or the frames folder that you used as an input for the extraction process

      • Leave any other options in this section blank as they are not required for this step.

        align10.jpg
        align10.jpg (37.35 KiB) Viewed 164603 times
    • Run
      Leave all other options at default. We're now ready to review our options and launch the manual tool.

      • You should end up with a screen similar to below:

        align11.jpg
        align11.jpg (219.33 KiB) Viewed 164603 times
      • Hit Alignments button to re-extract your faces.

    If you intend to train with a mask or use Warp to Landmarks, then copy the alignments file from your source frames location into your newly created faces folder.

Extracting a training set from an alignments file


Now you've cleaned up your alignments file, you may want to pull some of these faces out to use for a traning set. This is a simple task

Navigate to the "Tools" tab then the "Alignments" sub-tab:

align1.jpg
align1.jpg (52.65 KiB) Viewed 164613 times
  • Job
    This is a list of all of the different alignments jobs that are available in the tool.

    • Select "extract":

      align9.jpg
      align9.jpg (47.2 KiB) Viewed 164603 times
  • Data
    The location of the resources we want to process.

    • Alignments File: select the alignments file that was generated when after cleaning in the last step. It will be located in the same folder as the video you extracted from, or within the folder of the images you extracted from.

    • Faces Folder: select an empty folder where you want the outputted faces to go.

    • Frames Folder: select the video or the frames folder that you used as an input for the extraction process

    • Leave any other options in this section blank as they are not required for this step.

      align10.jpg
      align10.jpg (37.35 KiB) Viewed 164603 times
  • Extract
    These are options for extracting faces from an alignments file

    • Extract Every N: This will depend on the number of frames per second your input is. However, for a 25fps video, sane values are between about 12 - 25 (i.e. every half second to a second). Anything less than that and you will probably end up with too many similar faces. It is worth bearing in mind how many sources you are extracting from for your training set, and how many faces you want to have in your final training set, as well as how long your source is. These will all vary on a case by case basis.

    • Size: This is the size of the image holding the extracted face. Currently no models support above 256px so leave this at default

    • Align Eyes: I find this option entirely pointless. The aligner already aligns faces. No further alignment should be required.

      align12.jpg
      align12.jpg (15.48 KiB) Viewed 164562 times
  • Run
    Leave all other options at default. We're now ready to review our options and extract our training set from our cleansed alignments file to our selected folder.

    • You should end up with a screen similar to below:

      align13.jpg
      align13.jpg (216.04 KiB) Viewed 164562 times
    • Hit Alignments button to extract the faces.

If you intend to train with a mask or use Warp to Landmarks, then copy the alignments file from your source frames location into your newly created faces folder.

Merging facesets for training


So you've got a load of faces ready for training, and you've got a load of alignments files for the frames that the faces come from. Can these be combined into one source for training? Sure they can!

Preparation
We have to do a little bit of preparation prior to merging alignment files:

  • Firstly copy all of the faces you want to train with from their respective folders into one folder (we'll call it "training" for reference).

  • Copy all of the alignments files that relate to the faces you've just selected from their source frames location into the new "training" folder.

Now, in the GUI, navigate to the "Tools" tab then the "Alignments" sub-tab:

align1.jpg
align1.jpg (52.65 KiB) Viewed 164613 times
  • Job
    This is a list of all of the different alignments jobs that are available in the tool.

    • Select "merge":

      align14.jpg
      align14.jpg (46.07 KiB) Viewed 164558 times
  • Data
    The location of the resources we want to process.

    • Alignments File: Navigate to your "training" folder, hold shift and highlight all of the alignments files you just copied to select them.

    • Faces Folder: select the "training" folder that you placed all your faces in

    • Leave any other options in this section blank as they are not required for this step.

      align15.jpg
      align15.jpg (31.55 KiB) Viewed 164558 times
  • Run
    Leave all other options at default. We're now ready to review our options and merge our alignments files.

    • You should end up with a screen similar to below:

      align16.jpg
      align16.jpg (211.14 KiB) Viewed 164558 times
    • Hit Alignments button to merge the files

Finishing up
When the process has completed:

  • Delete all of the alignments files that you previously copied into the "training" folder from the "training" folder.

  • Rename the generated alignment file inside your "training" folder to "alignments.fsa"

Your training set is ready

My word is final


Tags:

User avatar
JiangshanLee
Posts: 3
Joined: Sun Jul 28, 2019 6:08 am

Re: [Guide] Extraction - A Workflow

Post by JiangshanLee »

can not see photo in your direction


User avatar
torzdf
Posts: 1008
Joined: Fri Jul 12, 2019 12:53 am
Answers: 128
Has thanked: 28 times
Been thanked: 193 times

Re: [Guide] Extraction - A Workflow

Post by torzdf »

They are hosted on imgur. Is that blocked where you are?

I have updated the hosting, so hopefully it works now.

My word is final


User avatar
JiangshanLee
Posts: 3
Joined: Sun Jul 28, 2019 6:08 am

Re: [Guide] Extraction - A Workflow

Post by JiangshanLee »

It's works, thank you very much


User avatar
JiangshanLee
Posts: 3
Joined: Sun Jul 28, 2019 6:08 am

Re: [Guide] Extraction - A Workflow

Post by JiangshanLee »

Thank you very much for your great tutorial. I have not been able to run successfully because of my poor English. Can you post a video tutorial? thank you very much.


User avatar
torzdf
Posts: 1008
Joined: Fri Jul 12, 2019 12:53 am
Answers: 128
Has thanked: 28 times
Been thanked: 193 times

Re: [Guide] Extraction - A Workflow

Post by torzdf »

I'm unlikely to post a video, sorry. I just don't have the time to make it at the moment.

My word is final


User avatar
Vaca
Posts: 2
Joined: Thu Jul 18, 2019 10:42 pm
Has thanked: 1 time

Re: [Guide] Extraction - A Workflow

Post by Vaca »

When I go to the sort section my Backend shows options for Cpu and Opencl, but no GPU. Is Opencl the same as GPU or do I not have something setup correctly?


User avatar
kilroythethird
Posts: 20
Joined: Fri Jul 12, 2019 11:35 pm
Answers: 2
Has thanked: 2 times
Been thanked: 9 times

Re: [Guide] Extraction - A Workflow

Post by kilroythethird »

@Vaca: If you see opencl and CPU options in sort you are using a old faceswap version. Update via "tools" -> "check for updates. Should also be a good bit faster after the update,

that amd guy


User avatar
Sangui
Posts: 7
Joined: Mon Aug 19, 2019 11:28 am
Has thanked: 1 time

Re: [Guide] Extraction - A Workflow

Post by Sangui »

The step "Extracting a training set from an alignments file" seems to not working anymore in the last update.

I try it before and that worked well. But now the generated images are cropped and rotated, not usable at all.
Do you have the same trouble too ?

I use the same settings as the guide tell. No error in logs.


User avatar
Sativa60
Posts: 3
Joined: Fri Sep 27, 2019 2:23 pm

Re: [Guide] Extraction - A Workflow

Post by Sativa60 »

thank you so much for the guide, would have never got where i am without it. Now that I have all my faces fixed, how do I put my friends into the faces i extracted from Gilligans Island intro. This is where I really confused. TIA


User avatar
singeno
Posts: 2
Joined: Thu Nov 21, 2019 3:34 am
Has thanked: 1 time

Re: [Guide] Extraction - A Workflow

Post by singeno »

Hi, torzdf,

Thanks for the great guide!
I got an error when I was on the step "Merge".
MSG: ERROR More than one alignments file required for merging.
Before this step, I skipped the manual alignments, because the result I got was pretty good. And I only had one "alignments.fsa" file which has been created itself.


User avatar
bryanlyon
Site Admin
Posts: 496
Joined: Fri Jul 12, 2019 12:49 am
Answers: 41
Location: San Francisco
Has thanked: 3 times
Been thanked: 120 times
Contact:

Re: [Guide] Extraction - A Workflow

Post by bryanlyon »

You only need to merge if you have more than one alignments file that you want to combine. If you've only got the one, you can skip that section. If you have two that you need to join, you just put both files in the alignments textbox with a space separating them.


User avatar
Sxybeast
Posts: 1
Joined: Wed Dec 04, 2019 6:35 am
Has thanked: 1 time

Should I do this twice?

Post by Sxybeast »

Hello, do I need to do this twice? Once for the face in the video I want replaced, and again for the face im replacing it with? I don't see an explanation for the "A" and "B" input files mentioned in the training guide.


User avatar
torzdf
Posts: 1008
Joined: Fri Jul 12, 2019 12:53 am
Answers: 128
Has thanked: 28 times
Been thanked: 193 times

Re: [Guide] Extraction - A Workflow

Post by torzdf »

You need to do this for every face set you are extracting.

My word is final


User avatar
JCA
Posts: 1
Joined: Sun Dec 08, 2019 10:33 pm

Re: [Guide] Extraction - A Workflow

Post by JCA »

I can't find an alignment job called "extract". Maybe it was changed in the newer versions.

So I'm stuck at this tutorial.

Could anyone help?


User avatar
torzdf
Posts: 1008
Joined: Fri Jul 12, 2019 12:53 am
Answers: 128
Has thanked: 28 times
Been thanked: 193 times

Re: [Guide] Extraction - A Workflow

Post by torzdf »

Nope still there:

Image 3.png
Image 3.png (21.26 KiB) Viewed 148823 times

My word is final


User avatar
tokafondo
Posts: 7
Joined: Mon Dec 16, 2019 1:43 pm

Re: [Guide] Extraction - A Workflow

Post by tokafondo »

Hi and thanks a lot for faceswap.

I'm following your guide, and I'd like to know if when fixing aligments, is there any option to just drag landmarks and/or bounding box instead of redetecting it. Also, it could be useful somekind resizing and rotating.

That way, I could copy the landmarks from a previous frame and then drag it to the desired position, and also resizing or rotating it, with no modification of the landmarks themselves.

Thanks.


User avatar
bryanlyon
Site Admin
Posts: 496
Joined: Fri Jul 12, 2019 12:49 am
Answers: 41
Location: San Francisco
Has thanked: 3 times
Been thanked: 120 times
Contact:

Re: [Guide] Extraction - A Workflow

Post by bryanlyon »

No, this is really not how the alignments are used. Even when copying from a previous frame, we recommend just using that as the starting point of detecting so that you get valid alignments for the face, otherwise you are just adding bad data that will confuse the model.


User avatar
tokafondo
Posts: 7
Joined: Mon Dec 16, 2019 1:43 pm

Re: [Guide] Extraction - A Workflow

Post by tokafondo »

bryanlyon wrote: Mon Dec 16, 2019 3:49 pm

No, this is really not how the alignments are used. Even when copying from a previous frame, we recommend just using that as the starting point of detecting so that you get valid alignments for the face, otherwise you are just adding bad data that will confuse the model.

Thanks. I understand what you say. That 'copy-drag' thing is what I thought as the solution to have faces swapped in takes where the face appears from one side of the screen, when the camera is panning. The first frame would show the ear, the following one, the ear and the cheek, the next one, the ear, cheek and half the nose... you know what I mean, don't you?

I haven't been able to get face recognition get those right, so I thought that just getting the landmarks from a few frames later, and copy to the former, it could do the trick.


User avatar
dingdong420
Posts: 5
Joined: Sat Dec 21, 2019 3:43 am

Cleaning up

Post by dingdong420 »

I can not find directions on what makes a "clean" set of images. Could you eleborate on what should and shouldn't be deleted when cleaning up an extraction folder?


Post Reply