Would a B&W data set (on both sides) train faster than color?

Want to understand the training process better? Got tips for which model to use and when? This is the place for you


Forum rules

Read the FAQs and search the forum before posting a new topic.

This forum is for discussing tips and understanding the process involved with Training a Faceswap model.

If you have found a bug are having issues with the Training process not working, then you should post in the Training Support forum.

Please mark any answers that fixed your problems so others can find the solutions.

Locked
User avatar
cosmico
Posts: 95
Joined: Sat Jan 18, 2020 6:32 pm
Has thanked: 13 times
Been thanked: 35 times

Would a B&W data set (on both sides) train faster than color?

Post by cosmico »

If both side A and B were both black and white, would it result in faster training of the model compared to both data sets being color?

My reasoning behind asking the question is every time it tries to guess a slightly better version of the picture, one of the things it's doing is guessing the perfect color of pixel. Making them both black and white seems like it would reduce a variable since now the model would focus on guessing shades of gray for its pixels instead of shades of infinite colors. So it seems like it would make sense. What's the deal in reality?

User avatar
bryanlyon
Site Admin
Posts: 793
Joined: Fri Jul 12, 2019 12:49 am
Answers: 44
Location: San Francisco
Has thanked: 4 times
Been thanked: 218 times
Contact:

Re: Would a B&W data set (on both sides) train faster than color?

Post by bryanlyon »

Yes, and no. Even with B&W sources, our model expects color right now. Using B&W images may save a little bit of time training right now (probably negligibly however). Better would be to modify the model (and training code) to expect 1 channel images in order to give faster results.

User avatar
torzdf
Posts: 2649
Joined: Fri Jul 12, 2019 12:53 am
Answers: 159
Has thanked: 128 times
Been thanked: 622 times

Re: Would a B&W data set (on both sides) train faster than color?

Post by torzdf »

Yeah. I would imagine there would be zero benefit with our current models. We convert all images to 3 channel colour anyway, and then we go and colour augment those images too.

You'd need to set up a specific pipeline for B+W (as Bryan says)

My word is final

User avatar
cosmico
Posts: 95
Joined: Sat Jan 18, 2020 6:32 pm
Has thanked: 13 times
Been thanked: 35 times

Re: Would a B&W data set (on both sides) train faster than color?

Post by cosmico »

I have a follow up question along the same lines
If I'm understanding this correctly you said that its hypothetically possible for it be faster if there was a model which was set up only to be B&W which there isn't, so it wont be faster.

So if I went back to the videos and pictures that make up my B data set, and applied some photo filters and adjusted the contrast and brightness and exposure to all of it, with the intentions of making it as similar as possible to the face of the video I plan on converting it on to, would it speed it up(make it easier for the algorithm to guess better)? As opposed to what I have now which is my B data set is made up of data of all different levels of brightness, contrast, colors, tints and hues?

User avatar
torzdf
Posts: 2649
Joined: Fri Jul 12, 2019 12:53 am
Answers: 159
Has thanked: 128 times
Been thanked: 622 times

Re: Would a B&W data set (on both sides) train faster than color?

Post by torzdf »

Quicker? Maybe, Better? No.

The model benefits from variety as it helps it learn what things like "lighting" are. By starving it of data, you are making it harder to understand certain concepts.

Add to this that we augment color anyway, and limiting data in this way would not end up serving any really beneficial purpose.

My word is final

User avatar
thinkapplefour
Posts: 2
Joined: Tue Mar 30, 2021 10:03 am

Re: Would a B&W data set (on both sides) train faster than color?

Post by thinkapplefour »

Making them both black and white seems like it would reduce a variable since now the model would focus on guessing shades of gray for its pixels instead of shades of infinite colors.

User avatar
bryanlyon
Site Admin
Posts: 793
Joined: Fri Jul 12, 2019 12:49 am
Answers: 44
Location: San Francisco
Has thanked: 4 times
Been thanked: 218 times
Contact:

Re: Would a B&W data set (on both sides) train faster than color?

Post by bryanlyon »

Unfortunately that's an intuitive guess that doesn't prove out in the real world. In fact, having multiple colors can actually give the model a better chance of finding details to grab onto and get results faster. In the end, B&W is a poor optimization for speed.

Though if you have old data only available in B&W it can raise it's own kind of issues. In that situation it might be best to convert the other side to B&W as well, but it also might be a good idea to leave it in color and then desaturate it after convert.

Locked