Does Cv2-dnn extractor means inferior alignments ?

Want to know about the Faceswap's Face Extraction process? Got tips, ideas or just want to learn about how it all works? Then this is the place for you


Forum rules

Read the FAQs and search the forum before posting a new topic.

This forum is for discussing tips and understanding the process involved for Extracting and preparing face sets for training a model in Faceswap.

If you have found a bug are having issues with the Extraction process not working, then you should post in the Extract Support forum.

Please mark any answers that fixed your problems so others can find the solutions.

Locked
User avatar
police.bike
Posts: 22
Joined: Tue Jun 30, 2020 3:37 pm
Has thanked: 7 times
Been thanked: 5 times

Does Cv2-dnn extractor means inferior alignments ?

Post by police.bike »

While extract , it is mentioned that cv2-dnn is CPU only while Sf3d is recommended.

Does it mean we should not use cv2-dnn at all ? It seems fast. So, Does it mean the quality or alignments will be of inferior quality?

Is the only difference between these detectors/aligners are about finding a face or is there output quality aspects involved ?

by bryanlyon » Wed Jul 01, 2020 9:53 pm

There are differences between the different detectors and aligners, and in general CV2-DNN is the worst of the options for quality, but by all means use the one that you like the most. The alignments shouldn't be "worse" (and we do accomodate them to some degree anyway with random skewing) it just might mean that some faces are missed or have to be removed when you're cleaning your data that the other detectors or aligners might have handled better.

If you're very short on data you may want to actually run multiple extracts using each one once, but otherwise just use the one you want.

Go to full post
User avatar
bryanlyon
Site Admin
Posts: 793
Joined: Fri Jul 12, 2019 12:49 am
Answers: 44
Location: San Francisco
Has thanked: 4 times
Been thanked: 218 times
Contact:

Re: Does Cv2-dnn extractor means inferior alignments ?

Post by bryanlyon »

There are differences between the different detectors and aligners, and in general CV2-DNN is the worst of the options for quality, but by all means use the one that you like the most. The alignments shouldn't be "worse" (and we do accomodate them to some degree anyway with random skewing) it just might mean that some faces are missed or have to be removed when you're cleaning your data that the other detectors or aligners might have handled better.

If you're very short on data you may want to actually run multiple extracts using each one once, but otherwise just use the one you want.

User avatar
police.bike
Posts: 22
Joined: Tue Jun 30, 2020 3:37 pm
Has thanked: 7 times
Been thanked: 5 times

Re: Does Cv2-dnn extractor means inferior alignments ?

Post by police.bike »

------------- Did a bit of research myself by extracting and converting with all the 3 detectors. HERE are my findings:

Cv2-Dnn - Gave me lesser number of faces. As admin mentioned, we need to run multiple passes. However, sequence of images is not guaranteed. It still missed a lot of faces. On conversion, edges were blurry

MtCnn - gave slightly better results than the first. However, there are still missed faces. Edges were fine in this but flickering came. I guess its because of missed frames

Sf3d - the bid daddy. Had to wait some time before seeing the result. But flawless detector. More faces. Better conversion. Crisp finishing.

Locked