[Guide] Extraction - A Workflow

Want to know about the Faceswap's Face Extraction process? Got tips, ideas or just want to learn about how it all works? Then this is the place for you


Forum rules

Read the FAQs and search the forum before posting a new topic.

This forum is for discussing tips and understanding the process involved for Extracting and preparing face sets for training a model in Faceswap.

If you have found a bug are having issues with the Extraction process not working, then you should post in the Extract Support forum.

Please mark any answers that fixed your problems so others can find the solutions.

User avatar
torzdf
Posts: 1312
Joined: Fri Jul 12, 2019 12:53 am
Answers: 121
Has thanked: 42 times
Been thanked: 262 times

[Guide] Extraction - A Workflow

Post by torzdf »

NB: This guide was correct at time of writing, but things do change. I will try to keep it updated.

Last edited: 14 March 2021

Contents



Introduction


A lot of people get overwhelmed when they start out Faceswapping, and many mistakes are made. Mistakes are good. It's how we learn, but it can sometimes help to have a bit of an understanding of the processes involved before diving in.

In this post I will detail a workflow for Extraction. I'm not saying that this is the best workflow, or what you should do, but it works for me, and will hopefully give you a good jump off point for creating your own workflow.

I will be using the GUI for this guide, but the premise is exactly the same for the cli (all the options present in the GUI are availablle in the cli).

Why Extract?


At the highest level, extraction consists of three phases: detection, alignment and mask generation:

  • Detection - The process of finding faces within a frame. The detector scans an image and selects areas of the image that it thinks are faces.
  • Alignment - Finding the "landmarks" (explained later) within a face and orienting the faces consistently. This takes the candidates from the detector, and tries to work out where the key features (eyes, nose etc.) are on the potential face. It then attempts to use this information to align the face.
  • Mask Generation - Identify the parts of an aligned face that contains face and block out those areas that contain background/obstructions.

There are several plugins for each of these. Their pros and cons are detailed in the tooltips (for the GUI) or the help text (for the cli), so I won't be covering specifics.

Extracting serves 2 main purposes:

  1. Training: To generate a set of faces for training. These faces will also contain the alignment information and masks which are required for training the model. This information is saved in the .png metadata of the extracted face.
  2. Converting: To generate an alignments file and mask for converting your final frames. The alignment file contains information about where each face is in each frame so that the conversion process knows where to swap faces for any given frame. When generating an alignments file for converting, then the extracted faces are not actually required, but they are useful for cleaning the alignments file (more on this later).

There is technically a 3rd purpose which is when you are extracting for converting and want some of these faces for training as well. I will also cover this off.

Whilst the extracted faces are not required for convert (only the alignments file is), it is useful to have them so that we can clean up our dataset for the conversion process. Similarly, if extracting faces for a training set, then the original alignments file is not actually required. However, it is useful to keep the alignments file, along with the original source, as it is the "master" document, and allows you to re-extract faces from the source directly from the information contained within the alignments file.

The Alignments File
The alignments file holds information about all the faces it finds in each frame, specifically where the face is, and where the 68 point landmarks are:
Image
It also stores any masks that have been extracted for each face.

Do not underestimate the importance of the alignments file. It is the master document that contains information about the location of faces and any associated information for each face for a given source. Having a clean and up to date alignments file can save you time in the future, as you can easily re-generate face sets straight from the file.

The alignments file specifically holds the following information:

  • The location of each face in each frame
  • The 68 point landmarks of each face in each frame
  • The "components" and "extended" mask for each frame
  • Any other user generated masks
  • A small jpg thumbnail of the aligned face (used for the manual tool)

So now we know why we need to extract faces, how do we nail down a good workflow?

Extracting


The first step, whatever your reason for creating a faceset is to actually extract the faces from the frames.

  • Data
    This is where we select our input videos or images, and our output location.

    Image 1.jpg
    Image 1.jpg (80.71 KiB) Viewed 242374 times
    • Input Dir: First up we need a source. This can be set to either a video file (video icon) or to a folder containing a series of images (picture icon).

    • Output Dir: Next we need to tell the process where to save the extracted faces to. Click the folder icon next to select an output location.

    • I ignore the next few options, so you probably should too, but the explanation of each of them is below:

      • Alignments: This option is in case you want to specify a different name/location for the alignments file. It's a lot easier to leave this blank, and let the process save it out to the default location (which will be in the same location as the source frames/video).

  • Plugins
    The plugins are used for first detecting faces in an image, reading the faces landmarks to align the image and creating a mask using a variety of masking methods.

    extract_plugins.png
    extract_plugins.png (7.77 KiB) Viewed 85642 times
    • Detector, Aligner: Select the ones you want to use. At the time of writing the best detector is S3FD and the best aligner is FAN

    • Masker: Select which masker(s) you would like to use. Any masker selected here will generate a mask in addition to the landmark based masks. Multiple masks can be generated (by selecting more than 1 masker), however be aware that all of these maskers use the GPU, so the more maskers you add, the slower the extract speed will be. The landmark based masks (Components and Extended) are automatically generated during extraction and are detailed below so that you know how they perform.

      Each masker has different pros and cons. If you wish to use a Neural Network based mask for training or converting your frames then you can choose a mask here.

      I tend to leave all the maskers off if I'm generating a set for conversion, as I'll want to amend a lot of the landmarks first, then generate the masks after the faces have been fixed (using the mask tool). For training sets, I will pick the mask that I intend to use for training with. Note, that you can always add more masks to an alignments file with the Mask tool.

      • Unet-DFL - A Neural Network mask designed to provide smart segmentation of mostly frontal faces. The mask model has been trained by community members.

        mask_unet-dfl.png
        mask_unet-dfl.png (118.75 KiB) Viewed 247359 times
      • VGG-Clear - A Neural Network mask designed to provide smart segmentation of mostly frontal faces clear of obstructions.

        mask_vgg-clear.png
        mask_vgg-clear.png (137.98 KiB) Viewed 247359 times
      • VGG-Obstructed - designed to provide smart segmentation of mostly frontal faces. The mask model has been specifically trained to recognize some facial obstructions (hands and eyeglasses).

        mask_vgg-obstructed.png
        mask_vgg-obstructed.png (130.46 KiB) Viewed 247359 times
      • The following Landmarks based masks are automatically created:

        • Components - A mask generated from the 68 point landmarks found in the alignments phase. This is a multi-part mask built around the outside points of the face.

          mask_components.png
          mask_components.png (122.11 KiB) Viewed 247359 times
        • Extended - One of the main problems with the components mask is that it only goes to the top of the eyebrows. The Extended Mask is based off the components mask, but attempts to go further up the forehead to avoid the "double eyebrow" issue, (i.e. when both the source and destination eyebrows appear in the final swap.).

          mask_extended.png
          mask_extended.png (126.5 KiB) Viewed 247359 times
    • Normalization: This processes the image fed into the extractor to better find faces in difficult lighting conditions. Different normalization methods work with different datasets, but I find "hist" to be the best all-rounder. This will slightly slow extraction, but it can lead to better alignments:

    • Re Feed: This option will re-feed slightly adjusted detected faces back into the aligner, then average the final result. This helps reduce "micro-jitter". For training sets, I will leave it at zero. For generating alignments for convert I will put this on. The higher the number of re-feeds, the better, but each increment slows down the extraction speed. Set this is high as you feel you have the patience for, but anything above 9 is probably not worth it.
      The below image shows re-feed set to 8 (left) vs re-feed set to 0.:
      Image

    • The next few options can also be ignored. Again, I will detail what they do so you can make your own call:

      • Rotate Images: This is entirely unnecessary with all current detectors except for maybe CV2-DNN. The current detectors are perfectly capable of finding faces at any orientation, so this will just slow down your process for little or no gain.

  • Face Processing
    Options for face filtering and selection are defined here.

    Image 4.jpg
    Image 4.jpg (31.66 KiB) Viewed 238211 times
    • Min-size: I normally set this at a fairly low value, but above zero, to filter out anything that is clearly too small to be a face. It will, however, depend on the size of your input frames, and then the size of the faces within your frames:

    • I ignore the next few options, for the reasons stated below, but you can make your own call. An explanation of each of them is below:

      • Filter/nFilter, Ref Threshold: These options are for filtering in/out unwanted faces. I don't use it. I prefer to do this myself at the next stage, as I will always be more effective at filtering out faces than a machine is, and it doesn't take long. Filtering faces also severely slows down the extraction process.

  • Output
    Options for the output processing of faces.

    extract_output.png
    extract_output.png (2.91 KiB) Viewed 29009 times
    • Extract Every N: This is the first option that will change depending on the purpose of this extraction.

      • If you are extracting for convert, or you are extracting for convert AND will be using some of the faces for training, then leave this on 1 (i.e. extract from every frame)

      • If you are extracting for training only, then set a value that seems sane. This will depend on the number of frames per second your input is. However, for a 25fps video, sane values are between about 12 - 25 (i.e. every half second to a second). Anything less than that and you will probably end up with too many similar faces. It is worth bearing in mind how many sources you are extracting from for your training set, and how many faces you want to have in your final training set, as well as how long your source is. These will all vary on a case by case basis.

    • These next few options are generally fine at default settings, however explanations below:

      • Size: This is the size of the image holding the extracted face. Generally 512px will be fine for most models. This is the size of the full head extracted image. When training a 'sub-crop' of this image will be used depending on the centering you have chosen. For 'face' centering, the image size available to the model will be 384px. When training with 'legacy' centering, the image size available will be 308px.

      • Save interval: I don't bother with, but it's up to you.

      • Debug Landmarks: Draws landmarks on the extracted face. Only useful for debugging purposes.

  • Run
    We're now ready to review our options and run Extraction.

    • You should end up with a screen similar to below:

      Image 7.jpg
      Image 7.jpg (270.4 KiB) Viewed 238211 times
    • Hit Extract and wait for your faces to output and your alignments file to be generated.

Sorting


Now that we have extracted our faces, we need to clean up the dataset and the alignments file. The Extractor does a good job of getting faces, but it isn't perfect. It will have got some false positives, it will have failed to align some faces and it will have also extracted people who we don't want to swap. Most likely if you go to your faces folder you will have large sections of the output which look something like this:

Image-11.jpg
Image-11.jpg (24.87 KiB) Viewed 262554 times

Cleaning that up doesn't look like it will be fun! Fortunately we can make this easier. The quickest and easiest way to clean our dataset is to sort the faces into a meaningful order and then delete all the faces we don't want. The best sort method, by some distance, is "sort by face".

NB: Sorting by face is RAM intensive. It has to do a LOT of calculations. I have tested successfully sorting 22k faces on 8GB of RAM. If you have to sort more faces than can be held in RAM, then the process will automatically switch to a much slower method, so if you are RAM limited, then you may want to split your dataset into smaller subsets. Theoretically, the amount of RAM required is (n2 * 24) / 1.8 where n is the number of images. You will need to take any other RAM overhead in to account too (i.e. other programs running, the images loaded to RAM), but theoretically it means 30,000 images will take (300002 * 24) / 1.8 = 12,000,000,000 bytes or about 11GB.

Head over to the "Tools" tab in the GUI and then to the "Sort" sub-tab:

sort_main.jpg
sort_main.jpg (60.92 KiB) Viewed 168671 times
  • Data
    This is where we select the Faceset that we extracted in the previous step for sorting.

    • Input: Enter the folder which contains your extracted faces from the previous step.

    • Output: This should be empty. We want the faces to be sorted in place.

    • Alignments: This is the path to the alignments file created during the extract step. It will be in the same folder as the input video (vid_name_alignments.fsa) or inside the folder of any input images (alignments.fsa). We need this file as we need to zoom in on the face to get decent results. You can still sort faces without an alignments file, but the results will be vastly inferior

    sort_data.png
    sort_data.png (2.92 KiB) Viewed 85636 times
  • Sort Settings
    How we want to sort the faces and any other specific settings.

    • Sort By: Set this to "Face"

    • You can ignore any other options in this section

    sort_settings.png
    sort_settings.png (4.41 KiB) Viewed 85636 times
  • Output
    How we want our sorted faces to be output.

    • Final Process: Make sure this is set to "Rename"

    • Keep: Make sure this is unchecked.

    • You can leave all of the other options untouched as they don't have any effect on rename tasks.

    sort_output.jpg
    sort_output.jpg (34.07 KiB) Viewed 168671 times
  • Run
    Leave all other options at default. We're now ready to review our options and run Sort.

    • You should end up with a screen similar to below:

      sort_settings.jpg
      sort_settings.jpg (221.02 KiB) Viewed 168671 times
    • Hit Sort to start the sorting process.

The process will start reading through the faces building up an identity for each. It will then cluster the faces together by similarity. The actual clustering process can take a long time as it has to compute a lot of data. Unfortunately there is no visual feedback as to it's progress, so please be patient.

Once complete you should find 99% of the faces sorted together:

Image-24.jpg
Image-24.jpg (24.68 KiB) Viewed 232666 times

And all the junk sorted together as well:

Image-25.jpg
Image-25.jpg (24.9 KiB) Viewed 232666 times

Now all you have to do is scroll through the faces folder and delete those faces out that you do not wish to keep.

Clean the Alignments File


Now we have removed all of the faces we don't want and just have a set that we do, we need to clean the alignments file. Why? Because all the information about the unwanted faces is still in the file, and this is likely to cause us problems in future. Cleaning the alignments file with the integrated tool also has the added advantage of renaming our faces back to their original filename, so it's a win-win.

Navigate to the "Tools" tab then the "Alignments" sub-tab:

alignments_main.jpg
alignments_main.jpg (72.3 KiB) Viewed 168661 times
  • Processing
    This section allows us to choose what action we wish to perform, as well as setting any output processing. We're only interested in the Job section.

    • Job: This is a list of all of the different alignments jobs that are available in the tool.

      • Select "remove-faces"

    • Output: Ignore this section as the remove-faces job does not generate any output,

    align_job.png
    align_job.png (5.14 KiB) Viewed 85636 times
  • Data
    The location of the resources we want to process.

    • Alignments File: select the alignments file that was generated during the extraction process It will be located in the same folder as the video you extracted from, or within the folder of the images you extracted from.

    • Faces Folder: select the faces folder that you outputted faces into as part of the extraction process (the same folder as you used for sort).

    • Leave the other options blank in this section as they are not required.

    alignments_data.jpg
    alignments_data.jpg (34.11 KiB) Viewed 168661 times
  • Run
    Leave all other options at default. We're now ready to review our options and clean the alignments file

    • You should end up with a screen similar to below:

      alignments_settings.jpg
      alignments_settings.jpg (219.14 KiB) Viewed 168661 times
    • Hit Alignments and let the process complete.

Once done your faces will have been renamed back to their default names, and all of the unwanted faces will have been removed from your alignments file.

The process will have backed up your old alignments file and put it next to your newly created file in it's original location. It will have the same name as your cleansed alignments file but with "backup_<timestamp>" appended on to the end of it. If you are happy that your new alignments file is correct, you can safely remove this backup file.

At this point, if you are extracting for convert (or the set is going to be used for convert and training) then you can entirely delete your faces folder. None of these faces are required any more. If you ever need to re-generate your face set, then this can be done with the Alignments Tool (extract job).

Manually Fix Alignments


Ok, we've extracted our faces, we've cleaned out all the trash, surely we're done now? Well hold on there fella. Sure, you could move on now, but do you want an ok swap or do you want a great swap?

Manual is useful for the following tasks:

  • Removing any left over "multiple faces" in a frame

  • Adding any alignments to those frames that are missing them

  • Fixing misaligned frames

  • Manually editing any masks you wish to use

Depending on what we are extracting our dataset for will dictate what we want to do here. If we are purely extracting for a training set, we could feasibly skip this step altogether, although it is a good idea to review the existing alignments to make sure any masks get built correctly.

If extracting for convert, then at the absolute minimum we want to fix any multiple faces in frames, and any frames missing alignments. This alone will improve the final swap. Depending on how thorough you want to be, you can then fix any poor alignments

I won't go into full details on how to use the Manual Tool. That is a guide in itself, but should be quite intuitive, with tooltips for all of the available options.

  • Launch the Manual Tool
    To launch the Manual tool first navigate to the "Tools" tab then the "Manual" sub-tab:

    manual_main.jpg
    manual_main.jpg (68.12 KiB) Viewed 168656 times
    • Data
      This is where we specify the alignments file and video file/frames folder we wish to work with.

      • Alignments: select the alignments file that was generated during the extraction process It will be located in the same folder as the video you extracted from, or within the folder of the images you extracted from.

      • Frames: select the video or the frames folder that you used as an input for the extraction process

      manual_data.jpg
      manual_data.jpg (34.49 KiB) Viewed 168656 times
    • Options
      This are optional actions that can be run. You won't generally need to change these, but I'll list what they do in case you do need to utilize them.

      • Regen Thumbs: The manual tool stores a cache of thumbnails for populating the Faces Viewer part of the tool. If you have used the old manual tool with your alignments file, then it is possible that the thumbs may have gone out of sync with the actual extracted faces. This option regenerates the thumbnails so that they are correct with the data that exists within the alignments file.

      • Single Process: In order to speed up the generation of the thumbnail cache, the process launches several threads to read the images from the video file. On some videos this causes the process to hang. If this happens to you, then select this option to extract the thumbnail in a single thread.

      manual_options.jpg
      manual_options.jpg (16.07 KiB) Viewed 168656 times
    • Run
      Leave all other options at default. We're now ready to review our options and launch the manual tool.

      • You should end up with a screen similar to below:

        manual_settings.jpg
        manual_settings.jpg (153.57 KiB) Viewed 168656 times
      • Hit the Manual button to launch the manual tool.

      • NB: It may take a minute or two for the Manual Tool to launch, as it relies on your GPU, so needs to initialize the aligners/maskers that it will use.

  • Manually Fixing for Converting
    For fixing for convert we want to do the following:

    • Multi-Faces

      • Change the filter mode to "Multiple Faces" in the filter pull down located between the Frames and Faces Viewer

      • You can right click faces in the Faces Viewer to delete faces you wish to delete

      • Alternatively, you can change the Editor to "Bounding Box" or "Extract Box" on the left-hand button bar and right click faces you want to delete from the Frame Viewer.

      • Use the transport buttons or "Z" and "X" to navigate to the previous or next frame with multiple faces.

      • Press "Ctrl+S" to save out your amended alignments file, or press the save icon located between the Frames and Faces Viewer.

    • Frames without faces

      • Change the filter mode to "No Faces" in the filter pull down located between the Frames and Faces Viewer

      • Change the edit mode to "Bounding Box" editor using the left=hand button bar

      • Use the transport buttons or "Z" and "X" to navigate to the previous or next frame without faces.

      • If you hit a frame where no face was detected but should have been, you can either click on the face and adjust the bounding box with the mouse, or you can copy the previous/next frame's alignments using the copy buttons on the left-hand side or using the "c" or "v" keys and edit those.

      • Repeat until your reach the last frame

      • Press "Ctrl+S" to save out your amended alignments file, or press the save icon located between the Frames and Faces Viewer.

    • Fixing alignments

      • Change the filter mode to "Has Faces" in the filter pull down located between the Frames and Faces Viewer

      • You can navigate through the frames with the transport controls, or click on a face in the Faces Viewer to jump straight to any frames that need attention.

      • If the alignments look incorrect, use the appropriate editor from the left-hand button bar to fix them up

      • Repeat until you reach the last frame

      • Press "Ctrl+S" to save out your amended alignments file, or press the save icon located between the Frames and Faces Viewer.

    A backup of your alignments file will have been created each time you saved during manual adjustment. If you are happy that your new alignments file is correct, you can delete these backups. They will be located in the same folder as the original alignments file.

  • Manually Fixing for a Training Set
    For fixing for training we want to do the following:

    • Fixing alignments

      • Change the filter mode to "Has Faces" in the filter pull down located between the Frames and Faces Viewer

      • You can navigate through the frames with the transport controls, or click on a face in the Faces Viewer to jump straight to any frames that need attention.

      • If the alignments look incorrect, use the appropriate editor from the left-hand button bar to fix them up

      • Repeat until you reach the last frame

      • Press "Ctrl+S" to save out your amended alignments file, or press the save icon located between the Frames and Faces Viewer.

    A backup of your alignments file will have been created each time you saved during manual adjustment. If you are happy that your new alignments file is correct, you can delete these backups. They will be located in the same folder as the original alignments file.

Once the alignments are fixed, you will need to regenerate your faceset. This is because the face image will have changed due to moving the landmarks, so the training process will not be able to find the faces you amended.

Firstly delete your output faces folder. You don't want this around any more.

You can then hit the Folder button located next to the save icon to extract faces straight from the alignments file.

An alternative method is listed below, but if you have successfully extracted from the Manual Tool, you can jump straight to Extracting a training set from an alignments file

  • Next navigate to the "Tools" tab then the "Alignments" sub-tab:

    align1.jpg
    align1.jpg (52.65 KiB) Viewed 255678 times
    • Processing
      This section allows us to choose what action we wish to perform, as well as setting any output processing. We're only interested in the Job section.

      • Job: This is a list of all of the different alignments jobs that are available in the tool.

        • Select "Extract" job.

      • Output: Ignore this section as the extract job does not generate any output,

      align9.jpg
      align9.jpg (46.53 KiB) Viewed 255668 times
    • Data
      The location of the resources we want to process.

      • Alignments File: select the alignments file that was generated when after cleaning in the last step. It will be located in the same folder as the video you extracted from, or within the folder of the images you extracted from.

      • Faces Folder: select an empty folder where you want the outputted faces to go.

      • Frames Folder: select the video or the frames folder that you used as an input for the extraction process

      • Leave any other options in this section blank as they are not required for this step.

      align10.jpg
      align10.jpg (40.13 KiB) Viewed 255668 times
    • Run
      Leave all other options at default. We're now ready to review our options and launch the manual tool.

      • You should end up with a screen similar to below:

        align11.jpg
        align11.jpg (229.75 KiB) Viewed 255668 times
      • Hit Alignments button to re-extract your faces.

Extracting a training set from an alignments file


Now you've cleaned up your alignments file, you may want to pull some of these faces out to use for a traning set. This is a simple task

Navigate to the "Tools" tab then the "Alignments" sub-tab:

align1.jpg
align1.jpg (52.65 KiB) Viewed 255678 times
  • Processing
    This section allows us to choose what action we wish to perform, as well as setting any output processing. We're only interested in the Job section.

    • Job: This is a list of all of the different alignments jobs that are available in the tool.

      • Select "Extract" job.

    • Output: Ignore this section as the extract job does not generate any output,

    align9.jpg
    align9.jpg (46.53 KiB) Viewed 255668 times
  • Data
    The location of the resources we want to process.

    • Alignments File: select the alignments file that was generated when after cleaning in the last step. It will be located in the same folder as the video you extracted from, or within the folder of the images you extracted from.

    • Faces Folder: select an empty folder where you want the outputted faces to go.

    • Frames Folder: select the video or the frames folder that you used as an input for the extraction process

    • Leave any other options in this section blank as they are not required for this step.

      align10.jpg
      align10.jpg (40.13 KiB) Viewed 255668 times
  • Extract
    These are options for extracting faces from an alignments file

    • Extract Every N: This will depend on the number of frames per second your input is. However, for a 25fps video, sane values are between about 12 - 25 (i.e. every half second to a second). Anything less than that and you will probably end up with too many similar faces. It is worth bearing in mind how many sources you are extracting from for your training set, and how many faces you want to have in your final training set, as well as how long your source is. These will all vary on a case by case basis.

    • Size: This is the size of the image holding the extracted face. Currently no models support above 256px so leave this at default

    • Large: Enabling this option will only extract faces that have not been up-scaled to the output size. For example, if the extract size is set to 512px and a face is found in a frame is 480px, it will not be extracted. If it is 520px it will.

      align12.jpg
      align12.jpg (22.96 KiB) Viewed 255627 times
  • Run
    Leave all other options at default. We're now ready to review our options and extract our training set from our cleansed alignments file to our selected folder.

    • You should end up with a screen similar to below:

      align13.jpg
      align13.jpg (230.12 KiB) Viewed 255627 times
    • Hit Alignments button to extract the faces.

Repeat this process for as many sources as you wish to generate training sets for.
Once complete, you can place all of the facesets into the same folder. Your training set is ready

My word is final


Tags:

User avatar
JiangshanLee
Posts: 3
Joined: Sun Jul 28, 2019 6:08 am

Re: [Guide] Extraction - A Workflow

Post by JiangshanLee »

can not see photo in your direction


User avatar
torzdf
Posts: 1312
Joined: Fri Jul 12, 2019 12:53 am
Answers: 121
Has thanked: 42 times
Been thanked: 262 times

Re: [Guide] Extraction - A Workflow

Post by torzdf »

They are hosted on imgur. Is that blocked where you are?

I have updated the hosting, so hopefully it works now.

My word is final


User avatar
JiangshanLee
Posts: 3
Joined: Sun Jul 28, 2019 6:08 am

Re: [Guide] Extraction - A Workflow

Post by JiangshanLee »

It's works, thank you very much


User avatar
JiangshanLee
Posts: 3
Joined: Sun Jul 28, 2019 6:08 am

Re: [Guide] Extraction - A Workflow

Post by JiangshanLee »

Thank you very much for your great tutorial. I have not been able to run successfully because of my poor English. Can you post a video tutorial? thank you very much.


User avatar
torzdf
Posts: 1312
Joined: Fri Jul 12, 2019 12:53 am
Answers: 121
Has thanked: 42 times
Been thanked: 262 times

Re: [Guide] Extraction - A Workflow

Post by torzdf »

I'm unlikely to post a video, sorry. I just don't have the time to make it at the moment.

My word is final


User avatar
Vaca
Posts: 2
Joined: Thu Jul 18, 2019 10:42 pm
Has thanked: 1 time
Been thanked: 1 time

Re: [Guide] Extraction - A Workflow

Post by Vaca »

When I go to the sort section my Backend shows options for Cpu and Opencl, but no GPU. Is Opencl the same as GPU or do I not have something setup correctly?


User avatar
kilroythethird
Posts: 20
Joined: Fri Jul 12, 2019 11:35 pm
Answers: 1
Has thanked: 2 times
Been thanked: 10 times

Re: [Guide] Extraction - A Workflow

Post by kilroythethird »

@Vaca: If you see opencl and CPU options in sort you are using a old faceswap version. Update via "tools" -> "check for updates. Should also be a good bit faster after the update,

that amd guy


User avatar
Sangui
Posts: 7
Joined: Mon Aug 19, 2019 11:28 am
Has thanked: 1 time

Re: [Guide] Extraction - A Workflow

Post by Sangui »

The step "Extracting a training set from an alignments file" seems to not working anymore in the last update.

I try it before and that worked well. But now the generated images are cropped and rotated, not usable at all.
Do you have the same trouble too ?

I use the same settings as the guide tell. No error in logs.


User avatar
Sativa60
Posts: 3
Joined: Fri Sep 27, 2019 2:23 pm

Re: [Guide] Extraction - A Workflow

Post by Sativa60 »

thank you so much for the guide, would have never got where i am without it. Now that I have all my faces fixed, how do I put my friends into the faces i extracted from Gilligans Island intro. This is where I really confused. TIA


User avatar
singeno
Posts: 2
Joined: Thu Nov 21, 2019 3:34 am
Has thanked: 1 time

Re: [Guide] Extraction - A Workflow

Post by singeno »

Hi, torzdf,

Thanks for the great guide!
I got an error when I was on the step "Merge".
MSG: ERROR More than one alignments file required for merging.
Before this step, I skipped the manual alignments, because the result I got was pretty good. And I only had one "alignments.fsa" file which has been created itself.


User avatar
bryanlyon
Site Admin
Posts: 601
Joined: Fri Jul 12, 2019 12:49 am
Answers: 40
Location: San Francisco
Has thanked: 3 times
Been thanked: 156 times
Contact:

Re: [Guide] Extraction - A Workflow

Post by bryanlyon »

You only need to merge if you have more than one alignments file that you want to combine. If you've only got the one, you can skip that section. If you have two that you need to join, you just put both files in the alignments textbox with a space separating them.


User avatar
Sxybeast
Posts: 1
Joined: Wed Dec 04, 2019 6:35 am
Has thanked: 1 time

Should I do this twice?

Post by Sxybeast »

Hello, do I need to do this twice? Once for the face in the video I want replaced, and again for the face im replacing it with? I don't see an explanation for the "A" and "B" input files mentioned in the training guide.


User avatar
torzdf
Posts: 1312
Joined: Fri Jul 12, 2019 12:53 am
Answers: 121
Has thanked: 42 times
Been thanked: 262 times

Re: [Guide] Extraction - A Workflow

Post by torzdf »

You need to do this for every face set you are extracting.

My word is final


User avatar
JCA
Posts: 1
Joined: Sun Dec 08, 2019 10:33 pm

Re: [Guide] Extraction - A Workflow

Post by JCA »

I can't find an alignment job called "extract". Maybe it was changed in the newer versions.

So I'm stuck at this tutorial.

Could anyone help?


User avatar
torzdf
Posts: 1312
Joined: Fri Jul 12, 2019 12:53 am
Answers: 121
Has thanked: 42 times
Been thanked: 262 times

Re: [Guide] Extraction - A Workflow

Post by torzdf »

Nope still there:

Image 3.png
Image 3.png (21.26 KiB) Viewed 239888 times

My word is final


User avatar
tokafondo
Posts: 31
Joined: Mon Dec 16, 2019 1:43 pm
Has thanked: 10 times
Been thanked: 4 times

Re: [Guide] Extraction - A Workflow

Post by tokafondo »

Hi and thanks a lot for faceswap.

I'm following your guide, and I'd like to know if when fixing aligments, is there any option to just drag landmarks and/or bounding box instead of redetecting it. Also, it could be useful somekind resizing and rotating.

That way, I could copy the landmarks from a previous frame and then drag it to the desired position, and also resizing or rotating it, with no modification of the landmarks themselves.

Thanks.


User avatar
bryanlyon
Site Admin
Posts: 601
Joined: Fri Jul 12, 2019 12:49 am
Answers: 40
Location: San Francisco
Has thanked: 3 times
Been thanked: 156 times
Contact:

Re: [Guide] Extraction - A Workflow

Post by bryanlyon »

No, this is really not how the alignments are used. Even when copying from a previous frame, we recommend just using that as the starting point of detecting so that you get valid alignments for the face, otherwise you are just adding bad data that will confuse the model.


User avatar
tokafondo
Posts: 31
Joined: Mon Dec 16, 2019 1:43 pm
Has thanked: 10 times
Been thanked: 4 times

Re: [Guide] Extraction - A Workflow

Post by tokafondo »

bryanlyon wrote: Mon Dec 16, 2019 3:49 pm

No, this is really not how the alignments are used. Even when copying from a previous frame, we recommend just using that as the starting point of detecting so that you get valid alignments for the face, otherwise you are just adding bad data that will confuse the model.

Thanks. I understand what you say. That 'copy-drag' thing is what I thought as the solution to have faces swapped in takes where the face appears from one side of the screen, when the camera is panning. The first frame would show the ear, the following one, the ear and the cheek, the next one, the ear, cheek and half the nose... you know what I mean, don't you?

I haven't been able to get face recognition get those right, so I thought that just getting the landmarks from a few frames later, and copy to the former, it could do the trick.


User avatar
dingdong420
Posts: 5
Joined: Sat Dec 21, 2019 3:43 am

Cleaning up

Post by dingdong420 »

I can not find directions on what makes a "clean" set of images. Could you eleborate on what should and shouldn't be deleted when cleaning up an extraction folder?


Post Reply