separating re-feeds from false positives

[MaxHunter]

Is there a way to give re-feeds it's own separate filter folder rather than lumping them in with the identity folder? It makes it incredibly hard to separate the "multiples" from the true false positives. When you go to the Manual tool and find you have a thousand frames of the same face when you thought you were putting in false positives it makes you want to pull hair out.

[torzdf]

Not sure I understand. Can you explain a bit more.

Identity filtering happens after the alignment process (and therefore after re-feed has been performed), so I suspect the answer to your question is going to be 'no', but I may be wrong as I'm not sure what you are trying to achieve

[MaxHunter]

Hmmm... interesting.

If you go to the identity filter it will have multiples of the same frame of the same face. I was under the impression that those multiples were from the ReFeed/ReAlign the program is using to average out the final frame. (I have both Identity and Re-Align ticked off and use the nFilter quiet regularly in batch mode extraction.) However the program will sometimes also filter out frames I want, and it's incredibly hard to figure out which is which because of the numbering system, ie, [frame#]0_0, [frame#]1_0, etc. This makes it hard to even manually filter them out.

Last night I was working/experimenting on a longer form video (20+ minutes 29,000+ frames,) and I added what I thought were falsely filtered out faces from the identity filter, I ended up with (I don't remember the exact number) 6,000+ frames that were just multiples of the same face of the same frames that were correctly filtered, and maybe a few hundred that were falsely filtered. Of course you won't know which is which until you use the manual tool and by that time it's too late. This was the second time this happened for the same video, and I finally just gave up and moved on.

[torzdf]

You will never have the same face from the same frame output more than once, no matter what you do. The only way this might happen is if s3fd has detected multiple faces, but the aligner has aligned them the same (can happen with close together faces). Even then, though, it isn't the same face. It is 2 separate detections.

[MaxHunter]

Weird, because what you're saying can't happen seems to be happening regularly to me. When I get the time (it takes hours to extract this particular video) I'll re-extract and post.

[bryanlyon]

One time I've seen this sort of thing was when I was working with a .webm file. It turns out that webm has weird functionality and will often duplicate frames (up to 1000 frames per second) meaning that FS sees identical frames over and over. But even then we increment the frame number.

If this is what's happening to you, I suggest first converting the webm to an mp4 or other standardized video format before you extract.

[MaxHunter]

Thanks Bryan but they are all mp4s, but that makes me wonder if it's a codec or encoding issue.

I haven't said anything because I just figured this was part of the "plan" and how you guys want it to work. It happens so often I just ignore and throw them all away it wasn't until the other day I realized I was missing a lot of frames from this particular video and threw them all in from the identity folder to only find they were all near identical detections of the same actor. What's worse is because they are near identical I can't X+delete to quickly scroll and delete them as that will lead to deleting the wrong "box". It's manageable if it's only a few hundred, but this was in the thousands It's usually just one or two extra duplicate boxes, but I have seen the very rare four a couple times. This only started with the addition of the ID/aligner buttons - which is why I assumed they were throw-aways from the ReFeed aligner. .

I'm sure it must be user error but it is happening to me none the less.

[torzdf]

What are the faces names? I'm really not sure what is happening here, but there can only be one face for each detection. The filename is named 'video_<frame_no>_<face_no>.png'. A face can only have the same filename once. So either you are looking at different frames (frame_no changes) or you are looking at a different detected face (face_no changes)

[MaxHunter]

It happens fairly often so I will definitely post the next time.

[MaxHunter]

Okay, first of all with out adding the identity folder faces the stats were:

23,560 Frames
5691 Faces
14 Multiples

After adding an edited Identity Filter Folder Frames:

14,200 Faces, and Multiples of the same face jumped to 1,165.

It took some time because there is no information given in the manual tool on the original frame location within a windows folder because the frame information is at the end of the frame name. However I stumbled upon one by happenstance. What's interesting is there are three frames in the folder but only two show up in the manual tool.

The names are:
000242_001370(video name hidden for copyright reasons)023073_0.png

000242_001370(video name)023073_0_0.png

000242_001370(video name)023073_0_0_0.png

Maybe it should also be noted that I had to do a "face extract" from the alingment tool because of mismatch frames. I wonder if that could be a reason.

Hope this helps.

[bryanlyon]

When you do an additional extract, you have to make sure you do not extract into the same folder. It re-extracts all files from the alignments. This is probably the cause of your issues.

[torzdf]

These names are wrong:

023073_0_0.png
023073_0_0_0.png

I don't know what you are doing in your workflow, but FS does not put multiple zeros on the end.

[MaxHunter]

@bryanlyon I bet that's it!! I did do an additional extract because the numbers were so low, which in turn would probably give those extra zeros @torzdf is saying can't happen. This makes sense now.

Edit:. Just tested this and what you described is exactly what was happening.

That's good information to know, and maybe should be put into the workflow tutorial as well.

Also...sorry about discussing what I wasn't suppose to discuss. It's the story of my life, breaking the rules by trying not to break the rules.

[MaxHunter]

When you used "extract from faces" to do a new alignment file it didn't filter those out?

I think it should have filtered those into a "duplicate" folder when you create a new alignment file.