Search found 3 matches

by kamapisachihd
Sat Jun 27, 2020 7:53 pm
Forum: Cloud Discussion
Topic: [Guide] Vast.ai setup script
Replies: 14
Views: 45201

Re: [Guide] Vast.ai setup script

First of all thank you for sharing this. I have successfully started training my model on remote GPU with Vast.ai although I have noticed that the GPU capacity of the instance labelled on the site is not what you get. Its actually not even a fraction of the GPU capacity that is being offered. I have...
by kamapisachihd
Sat Jun 13, 2020 11:13 pm
Forum: Training Support
Topic: tensorflow.python.framework.errors_impl.InternalError: GPU sync failed
Replies: 1
Views: 2006

tensorflow.python.framework.errors_impl.InternalError: GPU sync failed

I have been training my first model successfully for last few days, although post 300K iteration I have been running in the problem where training crashes abruptly, on first look its CUDA/CUDNN related errors. So I reinstalled everything from scratch assuming there was some problem with code itself ...
by kamapisachihd
Sun Jun 07, 2020 12:45 am
Forum: Training Support
Topic: Training for a face vs scene / reusing a model
Replies: 3
Views: 3220

Re: Training for a face vs scene / reusing a model

Thanx @ tarzdf for bringing forward this query, I have same problem as I have trained model with multiple random faces of Person A and Person B. Once model has finally trained I have tried extracting faces from Target Clip of Person B and convert it with Model trained. Unfortunately conversion never...