Faceswap Forum

If both side A and B were both black and white, would it result in faster training of the model compared to both data sets being color?

My reasoning behind asking the question is every time it tries to guess a slightly better version of the picture, one of the things it's doing is guessing the perfect color of pixel. Making them both black and white seems like it would reduce a variable since now the model would focus on guessing shades of gray for its pixels instead of shades of infinite colors. So it seems like it would make sense. What's the deal in reality?

Yes, and no. Even with B&W sources, our model expects color right now. Using B&W images may save a little bit of time training right now (probably negligibly however). Better would be to modify the model (and training code) to expect 1 channel images in order to give faster results.

Yeah. I would imagine there would be zero benefit with our current models. We convert all images to 3 channel colour anyway, and then we go and colour augment those images too.

You'd need to set up a specific pipeline for B+W (as Bryan says)

I have a follow up question along the same lines
If I'm understanding this correctly you said that its hypothetically possible for it be faster if there was a model which was set up only to be B&W which there isn't, so it wont be faster.

So if I went back to the videos and pictures that make up my B data set, and applied some photo filters and adjusted the contrast and brightness and exposure to all of it, with the intentions of making it as similar as possible to the face of the video I plan on converting it on to, would it speed it up(make it easier for the algorithm to guess better)? As opposed to what I have now which is my B data set is made up of data of all different levels of brightness, contrast, colors, tints and hues?

Quicker? Maybe, Better? No.

The model benefits from variety as it helps it learn what things like "lighting" are. By starving it of data, you are making it harder to understand certain concepts.

Add to this that we augment color anyway, and limiting data in this way would not end up serving any really beneficial purpose.

Making them both black and white seems like it would reduce a variable since now the model would focus on guessing shades of gray for its pixels instead of shades of infinite colors.

Unfortunately that's an intuitive guess that doesn't prove out in the real world. In fact, having multiple colors can actually give the model a better chance of finding details to grab onto and get results faster. In the end, B&W is a poor optimization for speed.

Though if you have old data only available in B&W it can raise it's own kind of issues. In that situation it might be best to convert the other side to B&W as well, but it also might be a good idea to leave it in color and then desaturate it after convert.