When I run my training with the 1070 and 1070ti which are both at 8 gbs of vram, i get an error no matter what batch size I use saying I don't have the vram to run the training. I went to 2 batches and it still said I didn't have enough memory. I tried it with and without distributed option enabled. Both times it didn't work, However when I ran the training on just a single card at 6 batches it ended up working.
I am using the realface trainer at 128 input/output. which is why i understand the memory requirment is so high but what i dont understand is how having two cards is less memory than a single card. even on a lower batch size to the point where i cant run any batch.
Both cards are fine solo and they worked before running a different model but i cant seem to run this one with multi gpu.
The 70ti is connected to the 16xpci and the normal 70 is connected to the 4xpci. Which i get will slow the cards down but i dont get how its using more memory if that is the issue.
Any help would be greatly appreciated.