Alignment files Bug

[BrassNucks]

It happens when you run a conversion on an old (pre-2.4) alignments set. I just tried it now using the new codebase and an old (c. 2021) alignments file/vid pair.

It updates the alignments file first, then runs the conversion, failing on every frame.

Code: Select all

Setting Faceswap backend to NVIDIA
04/21/2024 13:38:33 INFO     Log level set to: INFO
04/21/2024 13:38:39 INFO     Reading alignments from: '\\B450M\face_vidslib\-removed-\file_01_alignments.fsa'
04/21/2024 13:38:39 INFO     Updating alignments file to version 2.4
04/21/2024 13:38:39 INFO     Writing alignments to: '\\B450M\face_vidslib\-removed-\file_01_alignments.fsa'
04/21/2024 13:38:41 INFO     Loading Model from Phaze_A plugin...
04/21/2024 13:38:41 INFO     Config item: 'freeze_layers' has been updated from '['visual']' to '['keras_encoder']'
04/21/2024 13:38:41 INFO     Using configuration saved in state file
04/21/2024 13:38:42 INFO     Mixed precision compatibility check (mixed_float16): OK\nYour GPU will likely run quickly with dtype policy mixed_float16 as it has compute capability of at least 7.0. Your GPU: NVIDIA GeForce RTX 3060, compute capability 8.6
04/21/2024 13:38:47 INFO     Loaded model from disk: 'D:\sample_data\models\_Phaze-A\-removed-\phaze_a.h5'
04/21/2024 13:38:48 INFO     Loading Writer from Ffmpeg plugin...
04/21/2024 13:38:48 INFO     Outputting to: '\\B550m\d\face_sample_data\video_src\vidout\deepfakes\phaze_a\-removed-\file_01_converted.mkv'
04/21/2024 13:38:48 INFO     Loading Mask from Mask_Blend plugin...
04/21/2024 13:38:48 INFO     Loading Color from Avg_Color plugin...
04/21/2024 13:38:48 INFO     Loading Scaling from Sharpen plugin...
No alignment found for file_01_000001.mkv, skipping
No alignment found for file_01_000002.mkv, skipping 

.. etc

So with some recent discoveries I figured I would run the mask update script I wrote yesterday to batch them all, and that threw a ton of similar errors:

Code: Select all

Setting Faceswap backend to NVIDIA
04/21/2024 13:57:06 INFO     Log level set to: INFO
04/21/2024 13:57:10 INFO     Reading alignments from: '\\B450M\face_vidslib\-removed-\file_01_alignments.fsa'
04/21/2024 13:57:10 INFO     Loading Mask from Bisenet_Fp plugin...
04/21/2024 13:57:10 INFO     Initializing BiSeNet - Face Parsing (Mask)...
04/21/2024 13:57:14 INFO     Initialized BiSeNet - Face Parsing (Mask) with batchsize of 12
04/21/2024 13:57:14 INFO     Generating masks
04/21/2024 13:57:14 WARNING  Skipping frame not in alignments file: 'file_01_000001.mkv'
04/21/2024 13:57:14 WARNING  Skipping frame not in alignments file: 'file_01_000002.mkv'
04/21/2024 13:57:14 WARNING  Skipping frame not in alignments file: 'file_01_000003.mkv'
04/21/2024 13:57:14 WARNING  Skipping frame not in alignments file: 'file_01_000004.mkv'

So I loaded the alignments and video into the Manual tool, and it all looked fine and worked as expected. The alignments were visible and correct. I made a simple change to one face (to enable the Save button) and saved it. Ran the mask update script and got the very same errors, 'Skipping frame not in alignments file'. Short of re-extracting I don't know what else to do in order to resurrect these older alignments.

[torzdf]

Ok, I have managed to recreate this with an alignments file over 2 years old,

Please update and hopefully it will work for you.

[BrassNucks]

I want to say that everything's great now but the Remove-Faces function is now crashing and causing other issues. It unceremoniously tossed 6 frames into a _duplicates subfolder, which i've never seen before but that's not the big issue.

It complained that -fr was deprecated and to use (-r, --faces_folder) but I had neither in my command, I had -c for the faces folder.

Here's the output of the remove-faces stage:

Code: Select all

Setting Faceswap backend to NVIDIA
04/21/2024 22:39:21 INFO     Log level set to: INFO
04/21/2024 22:39:24 INFO     [ALIGNMENT DATA]
04/21/2024 22:39:24 INFO     Reading alignments from: 'C:\face_vidslib\-removed-\vids\file_03_alignments.fsa'
04/21/2024 22:39:24 INFO     [FACES DATA]
04/21/2024 22:39:24 INFO     Loading file list from C:\face_vidslib\-removed-\vids\file_03

04/21/2024 22:39:28 INFO     [REMOVE FACES FROM ALIGNMENTS]
04/21/2024 22:39:28 INFO     1338 alignment(s) were removed from alignments file

04/21/2024 22:39:31 ERROR    Got Exception on main handler:
Traceback (most recent call last):
  File "C:\Users\-removed-\faceswap\lib\cli\launcher.py", line 225, in execute_script
    process.process()
  File "C:\Users\-removed-\faceswap\tools\alignments\alignments.py", line 239, in process
    self._run_process(args)
  File "C:\Users\-removed-\faceswap\tools\alignments\alignments.py", line 214, in _run_process
    tool.process()
  File "C:\Users\-removed-\faceswap\tools\alignments\alignments.py", line 319, in process
    job.process()
  File "C:\Users\-removed-\faceswap\tools\alignments\jobs_faces.py", line 359, in process
    self._update_png_headers()
  File "C:\Users\-removed-\faceswap\tools\alignments\jobs_faces.py", line 397, in _update_png_headers
    face.from_alignment(self._alignments.get_faces_in_frame(frame)[new_index])
IndexError: list index out of range
04/21/2024 22:39:31 CRITICAL An unexpected crash has occurred. Crash report written to 'C:\Users\-removed-\faceswap\crash_report.2024.04.21.223928090413.log'. You MUST provide this file if seeking assistance. Please verify you are running the latest version of faceswap before reporting

The files in the faces folder didn't get renamed, probably because it crashed before getting there. I'm going to completely delete that folder and try extracting/removing again without touching the Manual tool at all, to see if that's the root of the problem.

Results:

1) Extraction went without a problem as usual (3033 files extracted). Done.
2) Sort the faces folder with Tools > Sort > Face (value 3.0), no batch or grouping. Files are renamed properly and sorted. Done.
3) Removed images from the faces folder who i'm not interested in swapping. 1644 images remain in there.
4) Coarse tuning: Run the Remove-Faces function from within Alignments tab. I hold my breath and click Alignments. It works! Done.
5) Delete the backup alignments file and relax.

So the basic coarse tuning portion works well enough. The 1644 images are renamed and put back in chronological order.

So let's fine tune it now:

1) Load the (new) alignments and video file into Tools > Manual. Note that it rewrites the alignments file again even before I do anything in here.
2) Pick a few and fix them, delete a few multiple faces, and save again. Everything seems to be OK this time, no crashes. Some of the misalignments are so wild that I just don't know what do with those but that's another topic.

So after re-extracting them yet again it seemed to go OK through the whole workflow.

Testing:
1) Load up the Tools > Preview with the latest alignments file and video.
2) Launch it and find no bisenet-fp mask available. Only components, extended, and none.
3) Double check the extract settings and verify I did check Bisenet-fp, and it's also trained in my model. It's the only masker I use anyways.

So I don't know what stripped out the bisenet-fp masks, I'll have to go back and add them now.

[torzdf]

Thanks for the update.

Given that your issue did not happen again when you went back to the beginning, I'll assume that something occurred during this period of fixing, which has now been resolved. Hopefully the mask issue is just an oversight.

[BrassNucks]

I think you're right, those glitches may have been byproducts of some version mismatching. Doing a fresh run seems to work just fine now, thanks for looking into it.

[wongca]

Hi , i have updated it but still got a problem

----> 'reasons' frames were stored in the alignments file (from video) as the video name followed by the frame number followed by ".png".

Exactly what has been happened with mine and i have tried other videos , they all cannot locate it.

[torzdf]

Exactly what has been happened with mine and i have tried other videos , they all cannot locate it.

Firstly, 100% make sure you are up to date. If you definitely are, please could you zip up a video + alignments file that causes this issue, and PM me with a download link (I don't care what is in the files, they are just used to trigger the bug, and will be deleted immediately after).

Please also provide the exact steps, using the shared files, to cause the issue.

[Jacksonjunge]

Hey there,

i got the same issue.
Sometimes i can convert, sometimes not. Extract sometimes yes, sometimes not. I have a lot of content. So the Vids that i have extracted first time yesterday that works (mostly),
Its weird and random.
Thhe same video convert issue (skipping frames because not in the agliment file) first run works, second not -
i use faceswap since 2020 and i know the issues that comes out sometimes. Sometimes its crashed but then i make my work to fix the issue but now i have no idea what i can do longer to fix the problem.
Manuel tool was my first thougt too, i made a few changes to save the agliment file and nothing changes during convert task.
Mask tool doesnt work. (Skipped frames that not in aglimet file. run the Extract tool the video and it extracted nothing.
Manuel tool show the right faces.

i have over 500 videos with a clean agliment file. i cannot extract all new!!!! thats horrible.

i tested a short clip with and extractet again. with following options:
1 Singelprcess
2: skip existing faces (i used the old agliment file)
3: skip saving faces

so here the results:

Loading...
Setting Faceswap backend to NVIDIA
04/24/2024 17:30:40 INFO Log level set to: INFO
04/24/2024 17:30:43 INFO Loading Detect from S3Fd plugin...
04/24/2024 17:30:43 INFO Loading Align from Fan plugin...
04/24/2024 17:30:43 INFO Loading Recognition from Vgg_Face2 plugin...
04/24/2024 17:30:43 INFO Loading Mask from Components plugin...
04/24/2024 17:30:43 INFO Loading Mask from Extended plugin...
04/24/2024 17:30:44 INFO Loading Mask from Unet_Dfl plugin...
04/24/2024 17:30:44 INFO Loading Mask from Bisenet_Fp plugin...
04/24/2024 17:30:44 INFO Starting, this may take a while...
04/24/2024 17:30:44 INFO Output Directory: None

04/24/2024 17:30:44 INFO Reading alignments from: 'R:\DEEPFAKE\DATA B\VID TO EXTRACT\GIACO VIDS\ich-spiele peter_gitarrenkurs_6.00x_1920x1080_amq-10\ich-spiele peter_gitarrenkurs_6.00x_1920x1080_amq-10_alignments.fsa'
04/24/2024 17:30:44 INFO Initializing S3FD (Detect)...
04/24/2024 17:30:46 INFO Initialized S3FD (Detect) with batchsize of 7

04/24/2024 17:31:02 INFO Initializing FAN (Align)...
04/24/2024 17:31:12 INFO Initialized FAN (Align) with batchsize of 13

04/24/2024 17:31:23 INFO Initializing VGGFace2 (Recognition)...
04/24/2024 17:31:29 INFO Initialized VGGFace2 (Recognition) with batchsize of 18

04/24/2024 17:31:31 INFO Initializing Components (Mask)...
04/24/2024 17:31:31 INFO Initialized Components (Mask) with batchsize of 1

04/24/2024 17:31:33 INFO Initializing Extended (Mask)...
04/24/2024 17:31:33 INFO Initialized Extended (Mask) with batchsize of 1

04/24/2024 17:31:36 INFO Initializing U-Net (Mask)...
04/24/2024 17:31:41 INFO Initialized U-Net (Mask) with batchsize of 11

04/24/2024 17:31:44 INFO Initializing BiSeNet - Face Parsing (Mask)...
04/24/2024 17:31:46 INFO Initialized BiSeNet - Face Parsing (Mask) with batchsize of 10

04/24/2024 17:31:50 INFO Writing alignments to: 'R:\DEEPFAKE\DATA B\VID TO EXTRACT\GIACO VIDS\ich-spiele peter_gitarrenkurs_6.00x_1920x1080_amq-10\ich-spiele peter_gitarrenkurs_6.00x_1920x1080_amq-10_alignments.fsa'
04/24/2024 17:31:50 INFO -------------------------
04/24/2024 17:31:50 INFO Images found: 58
04/24/2024 17:31:50 INFO Faces detected: 114
04/24/2024 17:31:50 INFO -------------------------
04/24/2024 17:31:50 INFO Process Successfully Completed. Shutting Down...

its weird, because the manuel tool shows me only 58 faces - and thats right
but now here comes the convert results - convert task was succesfull. but....

Loading...
Setting Faceswap backend to NVIDIA
04/24/2024 17:33:31 INFO Log level set to: INFO

04/24/2024 17:33:35 INFO Reading alignments from: 'R:\DEEPFAKE\DATA B\VID TO EXTRACT\GIACO VIDS\ich-spiele peter_gitarrenkurs_6.00x_1920x1080_amq-10\ich-spiele peter_gitarrenkurs_6.00x_1920x1080_amq-10_alignments.fsa'
04/24/2024 17:33:35 INFO Loading Model from Realface plugin...

04/24/2024 17:33:39 INFO Loading Writer from Ffmpeg plugin...
04/24/2024 17:33:39 INFO Outputting to: 'D:\ich-spiele peter_gitarrenkurs_6.00x_1920x1080_amq-10_converted_1.mkv'

04/24/2024 17:33:39 INFO Loading Color from Avg_Color plugin...
04/24/2024 17:33:39 INFO Loading Scaling from Sharpen plugin...

04/24/2024 17:33:59 INFO -------------------------
04/24/2024 17:33:59 INFO Images found: 58
04/24/2024 17:33:59 INFO Faces detected: 57
04/24/2024 17:33:59 INFO -------------------------
04/24/2024 17:33:59 INFO Process Successfully Completed. Shutting Down...

i ve deletet the model line for the forum.

[Jacksonjunge]

Hard Facts (from my first post)

Extract:
04/24/2024 17:31:50 INFO Images found: 58
04/24/2024 17:31:50 INFO Faces detected: 114

Direct convert:

04/24/2024 17:33:59 INFO Images found: 58
04/24/2024 17:33:59 INFO Faces detected: 57

Strange!

Tomorrow I can upload the ZIP with the Video and agliment file. In most cases I stored the agliment file's during the process to get the finally clean Agliment. In this specific case I have 2 older agliment files and the final version.

If the solution far away, I take 10 other Datasets and check them out.

Note: before I looking for a solution in the Forum, I done almost everything to help myself. Everything works fine, before I updated faceswap Monday.

The error can be found with almost 100 percent certainty in this update. I had already had several errors after an update and sawed that the error was related to the update. In fact, the bugs were fixed later with another update. Once I had to completely delete Conda and all Dependence. Since then, I have been afraid of the updates and updates rather rarely. Every 3 months I manually check whether updates are available. My computer is not connected to the Internet. If I need Internet, it is via USB tethering. But thank you in general for your work! If the "Voice Cloning" function should come, everything is perfect.

[JDBAU]

I'm having the same problem.

Everything was fine until I updated via the app 2 days ago.

Now converting any of my models seems broken. "No alignment found for ********.MP4, skipping".

The app tells me its up to date if I try to update. I haven't tried reinstalling yet.

Has anyone come up with a fix or workaround?

Help!

[torzdf]

If anyone can provide me with a video, alignments file + steps to reproduce this issue, please do, as I have not been able to.

[Jacksonjunge]

I don’t think so. If you make a new Dataset then it works. Sometimes it does work with older Agliments and Datasets, sometimes not. In one case the convert tasks failed. Closed faceswap restart faceswap and load NOT the Project again. Let it blank and go to convert section.

And select everything manually. Model, source, agliment, location and so on. The train task should be blank. In two cases it works.

The error is currently not identified.

I'm having the same problem.

Everything was fine until I updated via the app 2 days ago.

Now converting any of my models seems broken. "No alignment found for ********.MP4, skipping".

The app tells me its up to date if I try to update. I haven't tried reinstalling yet.

Has anyone come up with a fix or workaround?

Help!

[Jacksonjunge]

If anyone can provide me with a video, alignments file + steps to reproduce this issue, please do, as I have not been able to.

Later i will! In Germany is sleeping time. But like you said „ not been able to“… that’s the crazy thing.

What do you think about this?

Extract:
04/24/2024 17:31:50 INFO Images found: 58
04/24/2024 17:31:50 INFO Faces detected: 114

Direct convert:

04/24/2024 17:33:59 INFO Images found: 58
04/24/2024 17:33:59 INFO Faces detected: 57

I just remembered that I canceled a video convert at 30% (it was a new clip) and I was able to play the video. This has never worked before. Only if you have specified the frame range. That's really good. Or is that a good bug?

[BrassNucks]

The issue hasn't been resolved, the only way right now for me to do conversions is with a freshly extracted file (detailed verbosely earlier) from the very latest updated faceswap..

The problem is compounded by the fact that as soon as convert sees an old alignments file, it updates it to v2.4 so it may be even more difficult to troubleshoot. I don't know if the glitching occurs during that conversion or not, but it claims to update the alignments version every single time I retry the same file. I would think it only needs to update that once.

I have hundreds of older alignments, some as early as 2 weeks ago, and some that are two years old, none of them convert now. All of them exhibit the same behavior about skipping files due to them not being in the alignments file. We can load the same files into the Manual tool and view them with false hope, even save them from there, but still converting will not work. We still see the dreaded :

Code: Select all

Setting Faceswap backend to NVIDIA
04/26/2024 03:52:16 INFO     Log level set to: INFO
04/26/2024 03:52:19 INFO     Reading alignments from: '\\-removed-\ja_extra_20151225_alignments.fsa'
04/26/2024 03:52:19 INFO     Updating alignments file to version 2.4
04/26/2024 03:52:19 INFO     Writing alignments to: '\\-removed-\ja_extra_20151225_alignments.fsa'
04/26/2024 03:52:19 INFO     Loading Model from Phaze_A plugin...
04/26/2024 03:52:19 INFO     Config item: 'freeze_layers' has been updated from '['visual']' to '['keras_encoder']'
04/26/2024 03:52:19 INFO     Using configuration saved in state file
04/26/2024 03:52:19 WARNING  Mixed precision compatibility check (mixed_float16): WARNING
Your GPU may run slowly with dtype policy mixed_float16 because it does not have compute capability of at least 7.0. Your GPU:
  NVIDIA GeForce GTX 1050 Ti, compute capability 6.1
See https://developer.nvidia.com/cuda-gpus for a list of GPUs and their compute capabilities.
If you will use compatible GPU(s) not attached to this host, e.g. by running a multi-worker model, you can ignore this warning. This message will only be logged once
04/26/2024 03:52:35 INFO     Loaded model from disk: '\\-removed-\phaze_a.h5'
04/26/2024 03:52:36 INFO     Loading Writer from Ffmpeg plugin...
04/26/2024 03:52:36 INFO     Outputting to: '\\-removed-\ja_extra_20151225_converted_1.mkv'
04/26/2024 03:52:36 INFO     Loading Mask from Mask_Blend plugin...
04/26/2024 03:52:36 INFO     Loading Color from Avg_Color plugin...
04/26/2024 03:52:36 INFO     Loading Scaling from Sharpen plugin...
No alignment found for ja_extra_20151225_000001.mkv, skipping
No alignment found for ja_extra_20151225_000002.mkv, skipping
No alignment found for ja_extra_20151225_000003.mkv, skipping
No alignment found for ja_extra_20151225_000004.mkv, skipping
No alignment found for ja_extra_20151225_000005.mkv, skipping
No alignment found for ja_extra_20151225_000006.mkv, skipping

How can I decrypt the alignments files to inspect what's different? I managed to rar up a couple of alignments files before faceswap overwrites them with whatever it's doing.

[Jacksonjunge]

The issue hasn't been resolved, the only way right now for me to do conversions is with a freshly extracted file (detailed verbosely earlier) from the very latest updated faceswap..

The problem is compounded by the fact that as soon as convert sees an old alignments file, it updates it to v2.4 so it may be even more difficult to troubleshoot. I don't know if the glitching occurs during that conversion or not, but it claims to update the alignments version every single time I retry the same file. I would think it only needs to update that once.

I have hundreds of older alignments, some as early as 2 weeks ago, and some that are two years old, none of them convert now. All of them exhibit the same behavior about skipping files due to them not being in the alignments file. We can load the same files into the Manual tool and view them with false hope, even save them from there, but still converting will not work. We still see the dreaded :

Code: Select all

Setting Faceswap backend to NVIDIA
04/26/2024 03:52:16 INFO     Log level set to: INFO
04/26/2024 03:52:19 INFO     Reading alignments from: '\\-removed-\ja_extra_20151225_alignments.fsa'
04/26/2024 03:52:19 INFO     Updating alignments file to version 2.4
04/26/2024 03:52:19 INFO     Writing alignments to: '\\-removed-\ja_extra_20151225_alignments.fsa'
04/26/2024 03:52:19 INFO     Loading Model from Phaze_A plugin...
04/26/2024 03:52:19 INFO     Config item: 'freeze_layers' has been updated from '['visual']' to '['keras_encoder']'
04/26/2024 03:52:19 INFO     Using configuration saved in state file
04/26/2024 03:52:19 WARNING  Mixed precision compatibility check (mixed_float16): WARNING
Your GPU may run slowly with dtype policy mixed_float16 because it does not have compute capability of at least 7.0. Your GPU:
  NVIDIA GeForce GTX 1050 Ti, compute capability 6.1
See https://developer.nvidia.com/cuda-gpus for a list of GPUs and their compute capabilities.
If you will use compatible GPU(s) not attached to this host, e.g. by running a multi-worker model, you can ignore this warning. This message will only be logged once
04/26/2024 03:52:35 INFO     Loaded model from disk: '\\-removed-\phaze_a.h5'
04/26/2024 03:52:36 INFO     Loading Writer from Ffmpeg plugin...
04/26/2024 03:52:36 INFO     Outputting to: '\\-removed-\ja_extra_20151225_converted_1.mkv'
04/26/2024 03:52:36 INFO     Loading Mask from Mask_Blend plugin...
04/26/2024 03:52:36 INFO     Loading Color from Avg_Color plugin...
04/26/2024 03:52:36 INFO     Loading Scaling from Sharpen plugin...
No alignment found for ja_extra_20151225_000001.mkv, skipping
No alignment found for ja_extra_20151225_000002.mkv, skipping
No alignment found for ja_extra_20151225_000003.mkv, skipping
No alignment found for ja_extra_20151225_000004.mkv, skipping
No alignment found for ja_extra_20151225_000005.mkv, skipping
No alignment found for ja_extra_20151225_000006.mkv, skipping

How can I decrypt the alignments files to inspect what's different? I managed to rar up a couple of alignments files before faceswap overwrites them with whatever it's doing.

Like I said! A new extraction works. But I have the same problem that I have so much aglimwnts and every time when you make convert or something else the agliment is updated again.

I trust that a solution is coming. I have work more than I thought so I can’t upload the example. Maybe on Sunday.

[torzdf]

The problem is compounded by the fact that as soon as convert sees an old alignments file, it updates it to v2.4 so it may be even more difficult to troubleshoot. I don't know if the glitching occurs during that conversion or not, but .

I cannot recreate this. I have tried old alignment files, new alignment files, any and all alignment files. All work fine.

The only way I am going to be able to fix this is if someone provides me a source and alignment file. I have asked for this 3 times in this thread now, and no one has been forthcoming.

please anyone provide me with a source + alignments file that causes this issue, and most likely it will be a trivial fix

it claims to update the alignments version every single time I retry the same file. I would think it only needs to update that once

This definitely should not happen, so even one of these 'updated' alignments files may suffice.

[torzdf]

I trust that a solution is coming.

Not unless someone actually makes the effort to provide me with some data, it's not.

[Jacksonjunge]

I trust that a solution is coming.

Not unless someone actually makes the effort to provide me with some data, it's not.

Sorry but with full time job I have not much time. Like I said in the post before I will do it on Sunday.

[BrassNucks]

Here's one I had in the archives, it's from 2022 and loads just fine into the Manual tool. You can save the alignments after making changes to them, and it will still exhibit all the problems we're describing.

https://filebin.net/3j5vk9bc1x3xrcq3

[Jacksonjunge]

Here's one I had in the archives, it's from 2022 and loads just fine into the Manual tool. You can save the alignments after making changes to them, and it will still exhibit all the problems we're describing.

https://filebin.net/3j5vk9bc1x3xrcq3

I will check it too later. What OS and what Graphiccard do you have?