Are Iterations the same?
Posted: Sat Jan 28, 2023 5:42 pm
This is probably a basic question, but are all Iterations the same?
What I mean by this is I noticed my iterations slow down the higher my batch size is. For example, a batch size of 1 may produce 60k ITs over 6 hours, however a batch size of 4 of the same model will produce 25k ITs over the same amount of time. At first I thought there was a problem with having multiple gpus, but then I thought the BS 1 is seeing 120k faces (2 faces per iteration), whereas BS 4 is actually seeing 200k faces (8 faces per iteration) over the same amount of time. So judging by ITs can be a bit misleading. Is that correct? Wouldn't having a face count rather than IT count be more standardized? (Sorry for such a basic question.)