Hello, Just got this crash during training: (crash log attached)
Here is also the Console Output
I noticed two things:
1) The secondary window with the previews was gone when this happened.
2) I just rehit "start" and everything restarted flawlessly: no need to close faceswap and restart it.
THNX
//************************************************************************************************ Console Output
04/27/2020 15:26:30 ERROR Caught exception in thread: 'training_0'
04/27/2020 15:26:39 ERROR Got Exception on main handler:
Traceback (most recent call last):
File "C:\Users\fulvi\faceswap\lib\cli\launcher.py", line 155, in execute_script
process.process()
File "C:\Users\fulvi\faceswap\scripts\train.py", line 161, in process
self.end_thread(thread, err)
File "C:\Users\fulvi\faceswap\scripts\train.py", line 201, in end_thread
thread.join()
File "C:\Users\fulvi\faceswap\lib\multithreading.py", line 121, in join
raise thread.err[1].with_traceback(thread.err[2])
File "C:\Users\fulvi\faceswap\lib\multithreading.py", line 37, in run
self.target(self.args, **self.kwargs)
File "C:\Users\fulvi\faceswap\scripts\train.py", line 226, in _training
raise err
File "C:\Users\fulvi\faceswap\scripts\train.py", line 216, in training
self.run_training_cycle(model, trainer)
File "C:\Users\fulvi\faceswap\scripts\train.py", line 305, in run_training_cycle
trainer.train_one_step(viewer, timelapse)
File "C:\Users\fulvi\faceswap\plugins\train\trainer\_base.py", line 316, in train_one_step
raise err
File "C:\Users\fulvi\faceswap\plugins\train\trainer\_base.py", line 283, in train_one_step
loss[side] = batcher.train_one_batch()
File "C:\Users\fulvi\faceswap\plugins\train\trainer\_base.py", line 424, in train_one_batch
loss = self.model.predictors[self.side].train_on_batch(model_inputs, model_targets)
File "C:\Users\fulvi\MiniConda3\envs\faceswap\lib\site-packages\keras\engine\training.py", line 1217, in train_on_batch
outputs = self.train_function(ins)
File "C:\Users\fulvi\MiniConda3\envs\faceswap\lib\site-packages\keras\backend\tensorflow_backend.py", line 2715, in call
return self.call(inputs)
File "C:\Users\fulvi\MiniConda3\envs\faceswap\lib\site-packages\keras\backend\tensorflow_backend.py", line 2675, in call
fetched = self.callable_fn(array_vals)
File "C:\Users\fulvi\MiniConda3\envs\faceswap\lib\site-packages\tensorflow_core\python\client\session.py", line 1472, in call
run_metadata_ptr)
tensorflow.python.framework.errors_impl.InternalError: 2 root error(s) found.
(0) Internal: Dst tensor is not initialized.
[[{{node training_1/Adam/Variable_24/read}}]]
[[training_1/Adam/add_179/_1129]]
(1) Internal: Dst tensor is not initialized.
[[{{node training_1/Adam/Variable_24/read}}]]
0 successful operations.
0 derived errors ignored.
04/27/2020 15:26:39 CRITICAL An unexpected crash has occurred. Crash report written to 'C:\Users\fulvi\faceswap\crash_report.2020.04.27.152630710481.log'. You MUST provide this file if seeking assistance. Please verify you are running the latest version of faceswap before reporting
Process exited.
