If both side A and B were both black and white, would it result in faster training of the model compared to both data sets being color?
My reasoning behind asking the question is every time it tries to guess a slightly better version of the picture, one of the things it's doing is guessing the perfect color of pixel. Making them both black and white seems like it would reduce a variable since now the model would focus on guessing shades of gray for its pixels instead of shades of infinite colors. So it seems like it would make sense. What's the deal in reality?