Delete/copy/paste/many - face alignments

Want to know about the Faceswap's Face Extraction process? Got tips, ideas or just want to learn about how it all works? Then this is the place for you


Forum rules

Read the FAQs and search the forum before posting a new topic.

This forum is for discussing tips and understanding the process involved for Extracting and preparing face sets for training a model in Faceswap.

If you have found a bug are having issues with the Extraction process not working, then you should post in the Extract Support forum.

Please mark any answers that fixed your problems so others can find the solutions.

Locked
User avatar
SirDeepsALot
Posts: 35
Joined: Sat May 28, 2022 7:10 am
Has thanked: 7 times
Been thanked: 2 times

Delete/copy/paste/many - face alignments

Post by SirDeepsALot »

TLDR: Is there a delete many, paste many to this process I am missing?

I have this happen A LOT when I'm going through my manual alignments adjustment of faces:

The face is incorrectly captured enough that it must be fixed. If I try and drag and reposition things, it makes things worse usually, this means moving face alignments 1 dot at at a time to the correct shape and even that's fine. What's incredibly time-consuming is:

  1. having to delete misaligned faces one at a time
  2. only being able to paste one fixed-face-alignment one at a time.

eg I fix the alignment of that face like this:

now she holds that position for 10 more frames, but I can't just go "paste, paste, paste" or even "paste 10". I have to nav to the next face, delete, paste, repeat. There are 17,000 frames in this alignment file - surely there's a better way?

User avatar
torzdf
Posts: 2651
Joined: Fri Jul 12, 2019 12:53 am
Answers: 159
Has thanked: 129 times
Been thanked: 622 times

Re: Delete/copy/paste/many - face alignments

Post by torzdf »

Unfortunately not at the moment. At some point I will revisit the manual tool to enhance it some more, but it is a particularly complex bit of code to maintain. Also, as I am the only contributor to the project at the moment, this is not currently top of my priority list (it is on my priority list, just not at the top). Maintaining Faceswap takes a lot of my free time at the moment, so I have to try to prioritize where I can.

My word is final

User avatar
SirDeepsALot
Posts: 35
Joined: Sat May 28, 2022 7:10 am
Has thanked: 7 times
Been thanked: 2 times

Re: Delete/copy/paste/many - face alignments

Post by SirDeepsALot »

Alright mate, appreciate you taking the time to reply.

User avatar
MaxHunter
Posts: 193
Joined: Thu May 26, 2022 6:02 am
Has thanked: 176 times
Been thanked: 13 times

Re: Delete/copy/paste/many - face alignments

Post by MaxHunter »

Totally feel your problem on this one. I have multiple projects going because I get so bogged down with the manual stuff as well, and I get bored and move on to other projects.

User avatar
363LS2GTO
Posts: 30
Joined: Fri Jul 08, 2022 7:06 pm
Has thanked: 1 time
Been thanked: 4 times

Re: Delete/copy/paste/many - face alignments

Post by 363LS2GTO »

I second a way to delete or change many files in the manual tool with one movement. I understand that automating a process that usually requires a human is complex and probably time consuming e.g. for multiple people / incorrect multiple faces in a scene. However, just having the delete button and arrow keys on the keyboard work so I can give the mouse a break would be a fantastic improvement.

I have learned that unless you have a 'perfect' video for the automatic tools, 90% of a good swap is manually adjusting landmarks, masks, deleting incorrectly identified faces, and getting rid of extra faces in the video.

If there are multiple people in any scene, every frame has to be edited manually. This adds up very quickly. The crop tool quickly become your best friend.

If the video is older, slightly blurry, shot at certain angles, etc, hundreds to thousands of frames have to be manually edited or deleted--even for videos as short as 5-10 minutes or less.

I have seen bright lights reflected from a shiny surface show up as faces on what would appear to be "perfect" videos for the software to detect. Sometimes the program thinks hands are faces.

I am not sure if my mouse will fail or if I will get carpal tunnel first.

I am a totally newbie at this, but I can already tell that this is the main drawback of the entire program unless there is something that i am missing.

User avatar
363LS2GTO
Posts: 30
Joined: Fri Jul 08, 2022 7:06 pm
Has thanked: 1 time
Been thanked: 4 times

Re: Delete/copy/paste/many - face alignments

Post by 363LS2GTO »

I would like to say that there is something I was missing.

I went back and reread the extract guide regarding automatic sorting by face.

I did this at first, but I guess I got lucky and most of the extracted faces were done properly on simple videos. With very little bad data to delete, I began skipping the automatic sorting and going straight to the manual tool as it was faster and more accurate.

Now that I have more varied sources, I began running into trouble with the limitations of the manual tool.

Regardless of my rookie mistake, I still think that the original poster has a great point about not being able to delete & copy / paste more than one image at a time being quite a limitation. Right next to CPU limited extracting / encoding.

User avatar
torzdf
Posts: 2651
Joined: Fri Jul 12, 2019 12:53 am
Answers: 159
Has thanked: 129 times
Been thanked: 622 times

Re: Delete/copy/paste/many - face alignments

Post by torzdf »

EDIT: I posted this reply prior to seeing the follow up message. I'll leave it here anyway.

363LS2GTO wrote: Tue Jul 26, 2022 8:34 am

I have learned that unless you have a 'perfect' video for the automatic tools, 90% of a good swap is manually adjusting landmarks, masks, deleting incorrectly identified faces, and getting rid of extra faces in the video.

This is definitely true, and I am always looking for ways to reduce manual processing time, but I am also always looking at ways to improve the core functionality too, so it just comes down to the amount of work 1 person can realistically do on a code base which has grown substantially larger than 1 person can handle. I would always welcome pull requests to improve functionality. Sadly they are rarely forthcoming.

As I said before, the manual tool is particularly complex. One day I will revisit it, sadly that day is not today as it will require a considerable time investment. This is version 2 of the manual tool. It is considerably more feature packed than the original manual tool. Prior to that, we didn't have anything at all to fix alignments.

If there are multiple people in any scene, every frame has to be edited manually. This adds up very quickly. The crop tool quickly become your best friend.

If the video is older, slightly blurry, shot at certain angles, etc, hundreds to thousands of frames have to be manually edited or deleted--even for videos as short as 5-10 minutes or less.

I have seen bright lights reflected from a shiny surface show up as faces on what would appear to be "perfect" videos for the software to detect. Sometimes the program thinks hands are faces.

A LOT of this can be fixed prior to even using the manual tool. The sort tool is your friend. You should remove non-target and misaligned faces before launching the tool. This can be done far quicker and easier by sorting by face, deleting all the incorrect images from your extract folder, removing those faces from the alignments file, and then moving on to the manual tool.

It won't completely eliminate required manual work, but it will reduce it significantly.

My word is final

User avatar
363LS2GTO
Posts: 30
Joined: Fri Jul 08, 2022 7:06 pm
Has thanked: 1 time
Been thanked: 4 times

Re: Delete/copy/paste/many - face alignments

Post by 363LS2GTO »

I am not sure if this needs a new post or not, but since we are discussing problems with the manual tool, I thought it would be relevant to bring up a topic that was a problem for me and I found at the same time as this thread.

Once you move on from the included basic masks (I like the results from Bisenet-FP), if you edit the landmarks in the manual tool you must then use the mask tool located under tools.

The description is located in this thread.

viewtopic.php?p=3721&hilit=manually+edit#p3721

Otherwise all of your work for the conversion video will be wasted and you will have even worse problems for your training set.

Masks not based on landmarks (anything other than the default masks) are not automatically regenerated when you edit the face. This means that if you do not regenerate the missing masks with the mask tool, you will have to spend a lot of time deleting individual faces for training as the faces that you manually edited will not have the mask for training and the program will not run.

You will either have to use a default mask or manually locate the offending faces in your training set if you want to use your chosen mask (VGG, Bisenet, etc), You could also regenerate your entire training set from your alignments files, but this would also require a lot of work.

If you still want to use your chosen mask with your current training set, the training program will tell you, one at a time, the offending face that needs to be removed from your faces training set.

For me the program began crashing every 15 seconds, then every 30 seconds, up to a minute, then two minutes as I deleted about 50 faces from my training set before I removed all of the offending mask-less faces.

I do not believe this is mentioned in the Extract guide (if it is I did not understand the importance) and caused problems for my second swap.

User avatar
torzdf
Posts: 2651
Joined: Fri Jul 12, 2019 12:53 am
Answers: 159
Has thanked: 129 times
Been thanked: 622 times

Re: Delete/copy/paste/many - face alignments

Post by torzdf »

Noted. Extract guide updated.

My word is final

User avatar
363LS2GTO
Posts: 30
Joined: Fri Jul 08, 2022 7:06 pm
Has thanked: 1 time
Been thanked: 4 times

Re: Delete/copy/paste/many - face alignments

Post by 363LS2GTO »

A couple more things to note for anyone following this thread.

It is important to realize that every time the alignments file is updated, it is saved to the originally named alignments.fsw file. The backup file is the numbered file (I believe it states in the extract guide that it is named backup but I have not seen this).

To me this is counter intuitive but easy to follow once you know the convention.

I made the mistake of trying to use windows timestamps to identify the most current alignment file. This will not always work and you should always go by the name convention. The script generated at the bottom of the GUI will always tell you which file is the backup and which is the current if you need to verify due to changing the alignments' file name, deleting the originally named alignments file, etc.

Also, you should not change the name of the video associated with the alignments file. Otherwise no faces will be found. You can always change the name of the video back to the original name and the video and alignments file will sync up again.

As I am gathering a collection of alignments files, I am finding it necessary to keep track of them. I keep a folder labeled "backup alignments."

It is good to keep these as when you make a mistake (such as using 'Extract' instead of Tools, Alignments, Extract to pull a face set for training), it is possible to revert and not lose all of the work you put into the manual editing.

I put these backup alignments files into the backup folder right away as keeping more than the correct and up to date alignments file in the same folder as the source video is confusing. If not now, it will be in the future.

I think an entire supplemental guide detailing the workflow leading up to training could be written (most of it centered around using the manual tool) in addition to a best practices on how to organize the file cabinet to avoid clutter and confusion after making two or three fakes.

User avatar
MaxHunter
Posts: 193
Joined: Thu May 26, 2022 6:02 am
Has thanked: 176 times
Been thanked: 13 times

Re: Delete/copy/paste/many - face alignments

Post by MaxHunter »

I know Torzdf hates the face filter, but I've found it quite helpful for the problems you've had. (Granted, I'm still learning the process) I've found the amount of time I spend deleting false positives and multiples was so consuming I went against the advise in the tutorials and started using the face filters, and believe it does save overall time, and if not, definitely frustration.

Another hint I've found useful, is using the "next frame" and "copy previous/next" button. However it would be quite helpful to have a mass copy/delete button.

Locked