Page 1 of 1

Strange Performance Limitation Observation (Windows)

Posted: Thu Jul 08, 2021 8:25 pm
by ianstephens

We are noticing something strange from a performance and hardware utilization standpoint.

In particular, let's talk about the converting process. We are running on Windows.

We have a machine with dual 12 core Xeons (24 physical cores). We also have 32GB RAM along with a GTX 3090 that provides 24GB of VRAM.

No matter what we do, we can't seem to use more than 50-55% of available system RAM during conversion. We have tried to modify/increase "jobs" but it always results in the same.

We first noticed this issue when we tried a 4K video conversion and the conversion process had an OOM error. It was puzzling as there was at least 50% available RAM. We then turned on "allow growth" which resolved the issue.

Our thinking for this error was that with "allow growth" disabled, the conversion process would allocate 100% VRAM and Windows would therefore not release any "shared" system RAM to the conversion process. Allow growth fixed this as the conversion only allocated what it needed and when it needed it and we could run the 4K conversion.

So even with "allow growth" enabled, although we no longer get OOM, we can never use more than 50-55% of our system RAM. No matter what we do.

It's as if the process/libs/python that runs the conversion is never able to allocate any RAM in the "shared" pool.

We'd love to max out the RAM, CPUs, GPUs but we feel as though we're not able to achieve maximum performance and something is limiting/hitting a bottleneck somewhere.

Any thoughts, suggestions, or ideas are greatly appreciated. I think the best thought we had would be to move over to Ubuntu :D


Re: Strange Performance Limitation Observation (Windows)

Posted: Thu Jul 08, 2021 10:00 pm
by ianstephens

P.s. allocated a spare SSD we have here and will install it in the machine tomorrow for dual boot with Ubuntu. It's my daughters birthday tomorrow so likely work the the evening - while the world is peaceful and sleeping :)

Will report on Ubuntu performance and if it's still affected by this issue.


Re: Strange Performance Limitation Observation (Windows)

Posted: Thu Jul 08, 2021 10:02 pm
by bryanlyon

System ram is not really critical importance. OOMs apply to vram and not to system ram.


Re: Strange Performance Limitation Observation (Windows)

Posted: Mon Jul 12, 2021 12:04 am
by torzdf

FWIW convert is not particularly well optimized.

Either way, It should never use all of your RAM. It's specifically built to try not to use it by only holding onto those frames that it is processing at any given time.