Not enough RAM to perform linkage clustering. Using vector clustering. This will be significantly slower.

The Extraction process failing on you, and you aren't getting an error back with clear instructions? Tell us about it here


Forum rules

Read the FAQs and search the forum before posting a new topic.

This forum is for reporting errors with the Extraction process. If you want to get tips, or better understand the Extract process, then you should look in the Extract Discussion forum.

Please mark any answers that fixed your problems so others can find the solutions.

Locked
User avatar
cloud7556
Posts: 7
Joined: Thu Nov 18, 2021 8:33 am
Has thanked: 2 times

Not enough RAM to perform linkage clustering. Using vector clustering. This will be significantly slower.

Post by cloud7556 »

11/24/2021 16:53:03 WARNING Not enough RAM to perform linkage clustering. Using vector clustering. This will be significantly slower. Free RAM: 8373MB. Required for linkage method: 700884MB

I tried to sort 200000 face images, but the system prompts that RAM is not enough. I know that if I wait all the time, the sorting will succeed, but is there a faster setting? Obviously, this RAM is far beyond the capacity of my computer.
Previously, I used AMD GPU to run faceswap, and recently changed to NVIDIA.

User avatar
bryanlyon
Site Admin
Posts: 793
Joined: Fri Jul 12, 2019 12:49 am
Answers: 44
Location: San Francisco
Has thanked: 4 times
Been thanked: 218 times
Contact:

Re: Not enough RAM to perform linkage clustering. Using vector clustering. This will be significantly slower.

Post by bryanlyon »

Like it says, to use the faster clustering with that many images you'd need more than 700gb of RAM. It uses exponential amounts of RAM to cluster based on how many images you have. You can split it to subses of 50k images to sort it in 4 packs if you want to use the faster method.

User avatar
cloud7556
Posts: 7
Joined: Thu Nov 18, 2021 8:33 am
Has thanked: 2 times

Re: Not enough RAM to perform linkage clustering. Using vector clustering. This will be significantly slower.

Post by cloud7556 »

bryanlyon wrote: Wed Nov 24, 2021 4:50 pm

Like it says, to use the faster clustering with that many images you'd need more than 700gb of RAM. It uses exponential amounts of RAM to cluster based on how many images you have. You can split it to subses of 50k images to sort it in 4 packs if you want to use the faster method.

thank you! I got it!

Another question, I feel that the sorting speed of NVIDIA is not significantly improved compared with AMD. Is this normal? My CUDA has been running normally, but at present, faceswap has the following warnings. What are the negative effects?

11/24/2021 13:13:37 WARNING Limited tf.compat.v2.summary API due to missing TensorBoard installation.
11/24/2021 13:13:37 WARNING Limited tf.compat.v2.summary API due to missing TensorBoard installation.
11/24/2021 13:13:37 WARNING Limited tf.compat.v2.summary API due to missing TensorBoard installation.
11/24/2021 13:13:37 WARNING Limited tf.summary API due to missing TensorBoard installation.

Locked