is there a way to import mask made with XSeg? the mask that is made after extracting is bad and changing mask for every frame manually is a pain in the ass(specially if its 20000+ frames) plus sometimes you even cant do it as it shows error.
import a mask made with XSeg?
Read the FAQs and search the forum before posting a new topic.
This forum is for reporting errors with the Extraction process. If you want to get tips, or better understand the Extract process, then you should look in the Extract Discussion forum.
Please mark any answers that fixed your problems so others can find the solutions.
Re: import a mask made with XSeg?
Short answer, no. But would welcome PRs if people would find it useful.
However, we have not had complaints about the mask before so 2 questions:
- Which mask plugin are you using>?
- What does this mean?
sometimes you even cant do it as it shows error
. Can you be more specific?
My word is final
Re: import a mask made with XSeg?
torzdf wrote: ↑Tue Oct 25, 2022 5:45 pmShort answer, no. But would welcome PRs if people would find it useful.
However, we have not had complaints about the mask before so 2 questions:
- Which mask plugin are you using>?
- What does this mean?
sometimes you even cant do it as it shows error
. Can you be more specific?
i use bisenet-fp. 99% of the time it masks space that isnt face. if face is slim it masks as face areas around face. if face is bigger it for some reason masks not full face. and if i get to start a manual mask that means i need to manually change mask for ALOT of frames. in this case for 32+ thousands on destination and 3+ thousands on source.
with XSeg you can mask 15+ frames on source 15-20 frames on destination and let it work for an hour to get a perfect mask(but i cant use it with deepfacelab because for some reason python crashes when i try to use it.)
2) i get this error on most of the videos when i try to go to manual
10/25/2022 21:30:35 ERROR There is a mismatch between the number of frames found in the video file (32217) and the number of frames found in the alignments file (32218).
10/25/2022 21:30:35 ERROR This can be caused by a number of issues:
10/25/2022 21:30:35 ERROR - The video has a Variable Frame Rate and FFMPEG is having a hard time calculating the correct number of frames.
10/25/2022 21:30:35 ERROR - You are working with a Merged Alignments file. This is not supported for your current use case.
10/25/2022 21:30:35 ERROR You should either extract the video to individual frames, re-encode the video at a constant frame rate and re-run extraction or work with a dedicated alignments file for your requested video.
i reencode video. i recut video. i reextract faces and remask and no mater what i do i get this type of error on most of the videos. when i try to start manual in tools
the things i do for source and destination videos- i first extract, then go to tools>sort, then delete bad faces or no faces, then go to tools>alignments>remove-faces, then tools>mask to reapply the mask. then train.
Re: import a mask made with XSeg?
Your second point is super common, in fact, I literally just fixed the same issue.
I believe the answer is, no. You have to re-encode the movie. Personally I use, Handbrake (I believe it's still free.). I think Torz said it has something to do with it not being in a constant frame rate. I usually re-encode at 24 fps at a constant frame rate with Handbrake, then re-extract because I don't know if the alignment files will still match. You can also use the ffmpeg tool to extract the frames delete the bad frame and then re-merge the frames into a single film.
Re: import a mask made with XSeg?
MaxHunter wrote: ↑Tue Oct 25, 2022 9:38 pmYour second point is super common, in fact, I literally just fixed the same issue.
I believe the answer is, no. You have to re-encode the movie. Personally I use, Handbrake (I believe it's still free.). I think Torz said it has something to do with it not being in a constant frame rate. I usually re-encode at 24 fps at a constant frame rate with Handbrake, then re-extract because I don't know if the alignment files will still match. You can also use the ffmpeg tool to extract the frames delete the bad frame and then re-merge the frames into a single film.
i did that. i tried megui i tried xvid4psp and even davinci. the video is constant 30 fps.
Width : 1 920 pixels
Height : 1 080 pixels
Display aspect ratio : 16:9
Frame rate mode : Constant
Frame rate : 30.000 FPS
Color space : YUV
but its not only with this video. but still even if that worked manually remask about 36000 frames is insane.
i really hope devs will think about implementing XSeg.
Re: import a mask made with XSeg?
I find this very strange, and not something I've seen reported or experienced before. As a general mask, bisenet-fp is not perfect, but it is generally very good. If you can post some images to demonstrate what you mean, that may be useful to getting to a resolution.
2) i get this error on most of the videos when i try to go to manual
10/25/2022 21:30:35 ERROR There is a mismatch between the number of frames found in the video file (32217) and the number of frames found in the alignments file (32218).
10/25/2022 21:30:35 ERROR This can be caused by a number of issues:
10/25/2022 21:30:35 ERROR - The video has a Variable Frame Rate and FFMPEG is having a hard time calculating the correct number of frames.
10/25/2022 21:30:35 ERROR - You are working with a Merged Alignments file. This is not supported for your current use case.
10/25/2022 21:30:35 ERROR You should either extract the video to individual frames, re-encode the video at a constant frame rate and re-run extraction or work with a dedicated alignments file for your requested video.
i reencode video. i recut video. i reextract faces and remask and no mater what i do i get this type of error on most of the videos. when i try to start manual in tools
Yes, this happens sometimes. Working with video files is hard. Whilst the general faceswap process is ok with just iterating from first frame to last frame, the manual tool must have precise frame accuracy. Video is not built for precise frame accuracy and there is a distinct lack of standards that can cause this kind of issue (videos not cropped on keyframes, 3:2 pull-down, variable frame rates etc. etc.). I have put in many things to try to mitigate these issues, but there are always instances where this occurs. Ususally because of bad editing/encoding of the source video. Re-encoding to a known frame rate can help with this.
My word is final
Re: import a mask made with XSeg?
torzdf wrote: ↑Thu Oct 27, 2022 10:21 amI find this very strange, and not something I've seen reported or experienced before. As a general mask, bisenet-fp is not perfect, but it is generally very good. If you can post some images to demonstrate what you mean, that may be useful to getting to a resolution.
this the only 2 video that i managed to open on manual. <LINK REMOVED>
Yes, this happens sometimes. Working with video files is hard. Whilst the general faceswap process is ok with just iterating from first frame to last frame, the manual tool must have precise frame accuracy. Video is not built for precise frame accuracy and there is a distinct lack of standards that can cause this kind of issue (videos not cropped on keyframes, 3:2 pull-down, variable frame rates etc. etc.). I have put in many things to try to mitigate these issues, but there are always instances where this occurs. Ususally because of bad editing/encoding of the source video. Re-encoding to a known frame rate can help with this.
i tried reencoding in megui in xvid4psp and even in davinci resolve. not helping. most of the videos i have are 30 or 60fps.
i really hope XSeg will be implemented to this software. its easy to use and takes an hour or two to make a decent mask.
Re: import a mask made with XSeg?
On those images you posted, that is not the mask, it is the alignments (and they look broadly ok to me). The mask icon is the 5th one down on the left-hand side (and then you would need to select the 'bisenet-fp' mask from the right hand menu). The mask should show up as a red overlay
My word is final
Re: import a mask made with XSeg?
torzdf wrote: ↑Thu Oct 27, 2022 6:06 pmOn those images you posted, that is not the mask, it is the alignments (and they look broadly ok to me). The mask icon is the 5th one down on the left-hand side (and then you would need to select the 'bisenet-fp' mask from the right hand menu). The mask should show up as a red overlay
looks like this.
Re: import a mask made with XSeg?
Not sure what the issue is with that. That looks like it is masked perfectly. You have "include hair" set, but assuming that's what you want, I'm failing to see the problem.
My word is final
Re: import a mask made with XSeg?
I painted face. as you can see it looks like the source face went a bit smaller and tilted to the left. you can see a black outline of source face on the right and that it went on the hairs near forehead on the left. i went to 250000 iterations. it got chin perfect but then the applied face tilted. this can be fixed with manually redoing alignments and mask(i tried on a 10 seconds video(it took few hours to redo on each frame)) but if a video is more then 10 seconds its hell. this is 17 min. it will take days or weeks fixing frame by frame. with out option of mask training. the face type of video A is same as video B. they are pretty much same size, same width, same shape.
Re: import a mask made with XSeg?
Ok, this is why images are useful. The issue here isn't the bisenet-fp mask, per-se. It's a data/full face issue.
Most likely what is happening is that your "B" face has either a much narrower face than "A", or (more likely) your "B" face does not have enough variety in the training data and nearly always has hair framing the face. This hair is then 'learned" by the model (as it is always there in the data) and then gets represented in the swap. You may be able to solve this by adjusting the erosion settings in the 'preview' tool.
You can read more about this issue, why it occurs, and how you can potentially solve it here:
viewtopic.php?t=2078
My word is final