Color Space Question after reading the book

Discussions about research, Faceswapping and things that don't fit in the other categories here.


Locked
User avatar
Ryzen1988
Posts: 57
Joined: Thu Aug 11, 2022 8:31 am
Location: Netherlands
Has thanked: 8 times
Been thanked: 28 times

Color Space Question after reading the book

Post by Ryzen1988 »

So I just read the exploring deepfake books, very cool so left a good review.

But in there ofcourse was the resolution doubling -> vram & compute x 4 rule
The color space conversion rgb -> bgr and the fact that you do the whole convolutional shabang for each of the tree colours.
It already said that the mask itselfs is a lower resolution.

Since at least for me almost all faces come from video, and go back to video why isn't YUV 4:2:0 more used?
That would only use one layer at full resolution and the chroma luma at 25% the size.
Think of all the Vram and compute saved or the possibility to increase the output resolution without the heavy penalty.
I see not really any reason why the same convolutions and dense layers would not work with that instead of BGR :ugeek:

But probably people already tried this so why isn't this a option?

From what i have read YUV is not supported as such, but it seems fairly easy to map U & V to a grayscale layer and transform it back at the end.

Last edited by Ryzen1988 on Wed Aug 09, 2023 12:05 am, edited 1 time in total.
User avatar
torzdf
Posts: 2687
Joined: Fri Jul 12, 2019 12:53 am
Answers: 159
Has thanked: 135 times
Been thanked: 628 times

Re: Color Space Question after reading the book

Post by torzdf »

The short answer is, Faceswap has always been built to work with 8-bit BGR images. Video support was added much later in the life-cycle.

I have never experimented with other colour spaces for training, but, either way, to implement into Faceswap now would be a mammoth undertaking as it would impact every part of the codebase, so this kind of thing is unlikely to ever become a priority with the limited time I have available to develop.

My word is final

User avatar
Ryzen1988
Posts: 57
Joined: Thu Aug 11, 2022 8:31 am
Location: Netherlands
Has thanked: 8 times
Been thanked: 28 times

Re: Color Space Question after reading the book

Post by Ryzen1988 »

Oke, thats a very good reason for not doing that.

Thanks for the response

Locked