Where are the biases?

Discussions about research, Faceswapping and things that don't fit in the other categories here.


Locked
User avatar
Scrapemist
Posts: 16
Joined: Sat Nov 05, 2022 2:26 pm
Has thanked: 8 times

Where are the biases?

Post by Scrapemist »

So I'm coming from DFL and and now dipping my toes in FS, in prarticular the Phaze-A model.
To wrap my head around the meaning of all the settings I am studying how NN actual work.
This yt video is helping me alot to get a basic understanding.
It talks about the importance of weights and biases.
I see weights being represented in the settings of FS and talked about here on the forum.
But how do the biases come in to play?
Which settings adress them?

User avatar
torzdf
Posts: 2649
Joined: Fri Jul 12, 2019 12:53 am
Answers: 159
Has thanked: 128 times
Been thanked: 622 times

Re: Where are the biases?

Post by torzdf »

Biases are not directly exposed in settings. Biases are implemented at a layer by layer level, which is more granular than faceswap exposes. The actual biases (if used) are stored in the model .h5 file.

Faceswap does not generally use biases as they are not necessary for our functions (with the exception of dlight and realface model.... I would need to check with the plugin author why he chose to include them).

My word is final

User avatar
Scrapemist
Posts: 16
Joined: Sat Nov 05, 2022 2:26 pm
Has thanked: 8 times

Re: Where are the biases?

Post by Scrapemist »

I see. Thanks for clearing that up.

Then, just to see if I understand the NN correctly;
Is the first layer of nodes in the encoder the amount of pixels of the image times 3? (for each rgb channel)
So that each node contains an r, g or b value of a single pixel?

User avatar
bryanlyon
Site Admin
Posts: 793
Joined: Fri Jul 12, 2019 12:49 am
Answers: 44
Location: San Francisco
Has thanked: 4 times
Been thanked: 218 times
Contact:

Re: Where are the biases?

Post by bryanlyon »

Biases exist intrinsically in dense (Fully connected) layers but are generally not included in convolutional layers because convolutions work in a different way and don't really need them. HOWEVER, "weights" are generally taken to mean all variables used in the model. For example, convolutions include a kernel arrangement. That is saved in the "weights" file, but it's not actually a weight. Same with biases.

Basically, saying "Weights, Biases, Kernel initializes, attention values, and whatever else a given layer needs to operate" is just way too long (unless you're writing a full paper) and can just be (somewhat inaccurately) all be called "weights" together.

All our models use SOME biases, but some models use biases where they may typically be considered "optional".

Locked