Hello everybody,
i am from germany i am so sorry about my bad english...
At training with Faceswap i got the Error:
Code: Select all
MemoryError: Unable to allocate 15.0 MiB for an array with shape (60, 256, 256, 4) and data type uint8
07/19/2020 20:07:54 CRITICAL An unexpected crash has occurred. Crash report written to 'E:\Dokumente\system\faceswapInstallation\faceswap\crash_report.2020.07.19.200751596944.log'. You MUST provide this file if seeking assistance. Please verify you are running the latest version of faceswap before reporting
i try to train models on a AMD VEGA 56 8GB VRAM but i get everytime this error can anybody help me?
Status in Faceswap:
Code: Select all
Loading...
Setting Faceswap backend to AMD
07/19/2020 20:06:32 INFO Log level set to: INFO
07/19/2020 20:06:33 INFO Setting up for PlaidML
07/19/2020 20:06:33 INFO Setting GPU to largest available supported device. If you want to override this selection, run `plaidml-setup` from the command line.
07/19/2020 20:06:33 INFO Using GPU: ['opencl_amd_gfx900.0', 'opencl_amd_gfx900.0']
07/19/2020 20:06:33 INFO Successfully set up for PlaidML
Using plaidml.keras.backend backend.
07/19/2020 20:06:36 INFO Model A Directory: E:\Dokumente\system\Deepfake\PicA
07/19/2020 20:06:36 INFO Model B Directory: E:\Dokumente\system\Deepfake\PicB
07/19/2020 20:06:36 INFO Training data directory: E:\Dokumente\system\Deepfake\OutputVid
07/19/2020 20:06:36 INFO ===================================================
07/19/2020 20:06:36 INFO Starting
07/19/2020 20:06:36 INFO Press 'Stop' to save and quit
07/19/2020 20:06:36 INFO ===================================================
07/19/2020 20:06:37 INFO Loading data, this may take a while...
07/19/2020 20:06:37 INFO Loading Model from Original plugin...
07/19/2020 20:06:37 INFO No existing state file found. Generating.
07/19/2020 20:06:37 INFO Opening device "opencl_amd_gfx900.0"
07/19/2020 20:06:39 INFO Creating new 'original' model in folder: 'E:\Dokumente\system\Deepfake\OutputVid'
07/19/2020 20:06:39 INFO Loading Trainer from Original plugin...
07/19/2020 20:06:39 INFO Enabled TensorBoard Logging
07/19/2020 20:06:43 INFO Analyzing Ops: 162 of 520 operations complete
07/19/2020 20:07:04 INFO Analyzing Ops: 157 of 520 operations complete
07/19/2020 20:07:31 INFO Backing up models...
07/19/2020 20:07:32 INFO [Saved models] - Average since last save: face_loss_A: 0.20717, face_loss_B: 0.18052
07/19/2020 20:07:51 ERROR Caught exception in thread: '_training_0'
07/19/2020 20:07:54 ERROR Got Exception on main handler:
Traceback (most recent call last):
File "E:\Dokumente\system\faceswapInstallation\faceswap\lib\cli\launcher.py", line 155, in execute_script
process.process()
File "E:\Dokumente\system\faceswapInstallation\faceswap\scripts\train.py", line 161, in process
self._end_thread(thread, err)
File "E:\Dokumente\system\faceswapInstallation\faceswap\scripts\train.py", line 201, in _end_thread
thread.join()
File "E:\Dokumente\system\faceswapInstallation\faceswap\lib\multithreading.py", line 121, in join
raise thread.err[1].with_traceback(thread.err[2])
File "E:\Dokumente\system\faceswapInstallation\faceswap\lib\multithreading.py", line 37, in run
self._target(*self._args, **self._kwargs)
File "E:\Dokumente\system\faceswapInstallation\faceswap\scripts\train.py", line 226, in _training
raise err
File "E:\Dokumente\system\faceswapInstallation\faceswap\scripts\train.py", line 216, in _training
self._run_training_cycle(model, trainer)
File "E:\Dokumente\system\faceswapInstallation\faceswap\scripts\train.py", line 305, in _run_training_cycle
trainer.train_one_step(viewer, timelapse)
File "E:\Dokumente\system\faceswapInstallation\faceswap\plugins\train\trainer\_base.py", line 316, in train_one_step
raise err
File "E:\Dokumente\system\faceswapInstallation\faceswap\plugins\train\trainer\_base.py", line 283, in train_one_step
loss[side] = batcher.train_one_batch()
File "E:\Dokumente\system\faceswapInstallation\faceswap\plugins\train\trainer\_base.py", line 422, in train_one_batch
model_inputs, model_targets = self._get_next()
File "E:\Dokumente\system\faceswapInstallation\faceswap\plugins\train\trainer\_base.py", line 452, in _get_next
batch = next(self._feed)
File "E:\Dokumente\system\faceswapInstallation\faceswap\lib\multithreading.py", line 156, in iterator
self.check_and_raise_error()
File "E:\Dokumente\system\faceswapInstallation\faceswap\lib\multithreading.py", line 84, in check_and_raise_error
raise error[1].with_traceback(error[2])
File "E:\Dokumente\system\faceswapInstallation\faceswap\lib\multithreading.py", line 37, in run
self._target(*self._args, **self._kwargs)
File "E:\Dokumente\system\faceswapInstallation\faceswap\lib\multithreading.py", line 145, in _run
for item in self.generator(*self._gen_args, **self._gen_kwargs):
File "E:\Dokumente\system\faceswapInstallation\faceswap\lib\training_data.py", line 189, in _minibatch
yield self._process_batch(img_paths, side)
File "E:\Dokumente\system\faceswapInstallation\faceswap\lib\training_data.py", line 198, in _process_batch
batch = self._apply_mask(filenames, batch, side)
File "E:\Dokumente\system\faceswapInstallation\faceswap\lib\training_data.py", line 257, in _apply_mask
batch = np.concatenate((batch, masks), axis=-1)
File "<__array_function__ internals>", line 6, in concatenate
MemoryError: Unable to allocate 15.0 MiB for an array with shape (60, 256, 256, 4) and data type uint8
07/19/2020 20:07:54 CRITICAL An unexpected crash has occurred. Crash report written to 'E:\Dokumente\system\faceswapInstallation\faceswap\crash_report.2020.07.19.200751596944.log'. You MUST provide this file if seeking assistance. Please verify you are running the latest version of faceswap before reporting
Process exited.
Please write if you need the log file too.
Thank you...
