I installed the faceswap using the linux installer released on github. However, after the installation, the following errors were encountered during the extraction and training phases. Here is the crash report I get during training phase.
Code: Select all
06/04/2023 10:23:25 MainProcess MainThread logger log_setup INFO Log level set to: INFO
06/04/2023 10:23:28 MainProcess MainThread train _get_images INFO Model A Directory: '/data/Generation/faceswap/faces/01/001' (116 images)
06/04/2023 10:23:28 MainProcess MainThread train _get_images INFO Model B Directory: '/data/Generation/faceswap/faces/02/001' (148 images)
06/04/2023 10:23:28 MainProcess MainThread train _validate_image_counts WARNING At least one of your input folders contains fewer than 250 images. Results are likely to be poor.
06/04/2023 10:23:28 MainProcess MainThread train _validate_image_counts WARNING You need to provide a significant number of images to successfully train a Neural Network. Aim for between 500 - 5000 images per side.
06/04/2023 10:23:28 MainProcess MainThread train process INFO Training data directory: /data/Generation/faceswap/01_02_model
06/04/2023 10:23:28 MainProcess MainThread train _output_startup_info INFO ===================================================
06/04/2023 10:23:28 MainProcess MainThread train _output_startup_info INFO Starting
06/04/2023 10:23:28 MainProcess MainThread train _output_startup_info INFO Press 'ENTER' to save and quit
06/04/2023 10:23:28 MainProcess MainThread train _output_startup_info INFO Press 'S' to save model weights immediately
06/04/2023 10:23:28 MainProcess MainThread train _output_startup_info INFO ===================================================
06/04/2023 10:23:29 MainProcess _training train _training INFO Loading data, this may take a while...
06/04/2023 10:23:29 MainProcess _training plugin_loader _import INFO Loading Model from Original plugin...
06/04/2023 10:23:29 MainProcess _training model _load INFO No existing state file found. Generating.
06/04/2023 10:23:29 MainProcess _training settings get_mixed_precision_layers INFO Storing Mixed Precision compatible layers. Please ignore any following warnings about using mixed precision.
06/04/2023 10:23:29 MainProcess _training device_compatibility_check _log_device_compatibility_check INFO Mixed precision compatibility check (mixed_float16): OK\nYour GPU will likely run quickly with dtype policy mixed_float16 as it has compute capability of at least 7.0. Your GPU: NVIDIA GeForce RTX 3080 Ti, compute capability 8.6
06/04/2023 10:23:31 MainProcess _training plugin_loader _import INFO Loading Trainer from Original plugin...
06/04/2023 10:23:47 MainProcess MainThread train _end_thread CRITICAL Error caught! Exiting...
06/04/2023 10:23:47 MainProcess MainThread multithreading join ERROR Caught exception in thread: '_training'
06/04/2023 10:23:52 MainProcess MainThread launcher execute_script ERROR Got Exception on main handler:
Traceback (most recent call last):
File "/data/Generation/faceswap/lib/cli/launcher.py", line 230, in execute_script
process.process()
File "/data/Generation/faceswap/scripts/train.py", line 213, in process
self._end_thread(thread, err)
File "/data/Generation/faceswap/scripts/train.py", line 253, in _end_thread
thread.join()
File "/data/Generation/faceswap/lib/multithreading.py", line 220, in join
raise thread.err[1].with_traceback(thread.err[2])
File "/data/Generation/faceswap/lib/multithreading.py", line 96, in run
self._target(*self._args, **self._kwargs)
File "/data/Generation/faceswap/scripts/train.py", line 275, in _training
raise err
File "/data/Generation/faceswap/scripts/train.py", line 265, in _training
self._run_training_cycle(model, trainer)
File "/data/Generation/faceswap/scripts/train.py", line 353, in _run_training_cycle
trainer.train_one_step(viewer, timelapse)
File "/data/Generation/faceswap/plugins/train/trainer/_base.py", line 223, in train_one_step
loss: List[float] = self._model.model.train_on_batch(model_inputs, y=model_targets)
File "/data/Generation/environment/envs/faceswap/lib/python3.9/site-packages/keras/engine/training.py", line 2093, in train_on_batch
logs = self.train_function(iterator)
File "/data/Generation/environment/envs/faceswap/lib/python3.9/site-packages/tensorflow/python/util/traceback_utils.py", line 153, in error_handler
raise e.with_traceback(filtered_tb) from None
File "/data/Generation/environment/envs/faceswap/lib/python3.9/site-packages/tensorflow/python/eager/execute.py", line 54, in quick_execute
tensors = pywrap_tfe.TFE_Py_Execute(ctx._handle, device_name, op_name,
tensorflow.python.framework.errors_impl.UnimplementedError: Graph execution error:
Detected at node 'original/encoder/conv_128_1_conv2d/Conv2D' defined at (most recent call last):
File "/data/Generation/environment/envs/faceswap/lib/python3.9/threading.py", line 930, in _bootstrap
self._bootstrap_inner()
File "/data/Generation/environment/envs/faceswap/lib/python3.9/threading.py", line 973, in _bootstrap_inner
self.run()
File "/data/Generation/faceswap/lib/multithreading.py", line 96, in run
self._target(*self._args, **self._kwargs)
File "/data/Generation/faceswap/scripts/train.py", line 265, in _training
self._run_training_cycle(model, trainer)
File "/data/Generation/faceswap/scripts/train.py", line 353, in _run_training_cycle
trainer.train_one_step(viewer, timelapse)
File "/data/Generation/faceswap/plugins/train/trainer/_base.py", line 223, in train_one_step
loss: List[float] = self._model.model.train_on_batch(model_inputs, y=model_targets)
File "/data/Generation/environment/envs/faceswap/lib/python3.9/site-packages/keras/engine/training.py", line 2093, in train_on_batch
logs = self.train_function(iterator)
File "/data/Generation/environment/envs/faceswap/lib/python3.9/site-packages/keras/engine/training.py", line 1021, in train_function
return step_function(self, iterator)
File "/data/Generation/environment/envs/faceswap/lib/python3.9/site-packages/keras/engine/training.py", line 1010, in step_function
outputs = model.distribute_strategy.run(run_step, args=(data,))
File "/data/Generation/environment/envs/faceswap/lib/python3.9/site-packages/keras/engine/training.py", line 1000, in run_step
outputs = model.train_step(data)
File "/data/Generation/environment/envs/faceswap/lib/python3.9/site-packages/keras/engine/training.py", line 859, in train_step
y_pred = self(x, training=True)
File "/data/Generation/environment/envs/faceswap/lib/python3.9/site-packages/keras/utils/traceback_utils.py", line 64, in error_handler
return fn(*args, **kwargs)
File "/data/Generation/environment/envs/faceswap/lib/python3.9/site-packages/keras/engine/base_layer.py", line 1096, in __call__
outputs = call_fn(inputs, *args, **kwargs)
File "/data/Generation/environment/envs/faceswap/lib/python3.9/site-packages/keras/utils/traceback_utils.py", line 92, in error_handler
return fn(*args, **kwargs)
File "/data/Generation/environment/envs/faceswap/lib/python3.9/site-packages/keras/engine/functional.py", line 451, in call
return self._run_internal_graph(
File "/data/Generation/environment/envs/faceswap/lib/python3.9/site-packages/keras/engine/functional.py", line 589, in _run_internal_graph
outputs = node.layer(*args, **kwargs)
File "/data/Generation/environment/envs/faceswap/lib/python3.9/site-packages/keras/utils/traceback_utils.py", line 64, in error_handler
return fn(*args, **kwargs)
File "/data/Generation/environment/envs/faceswap/lib/python3.9/site-packages/keras/engine/base_layer.py", line 1096, in __call__
outputs = call_fn(inputs, *args, **kwargs)
File "/data/Generation/environment/envs/faceswap/lib/python3.9/site-packages/keras/utils/traceback_utils.py", line 92, in error_handler
return fn(*args, **kwargs)
File "/data/Generation/environment/envs/faceswap/lib/python3.9/site-packages/keras/engine/functional.py", line 451, in call
return self._run_internal_graph(
File "/data/Generation/environment/envs/faceswap/lib/python3.9/site-packages/keras/engine/functional.py", line 589, in _run_internal_graph
outputs = node.layer(*args, **kwargs)
File "/data/Generation/environment/envs/faceswap/lib/python3.9/site-packages/keras/utils/traceback_utils.py", line 64, in error_handler
return fn(*args, **kwargs)
File "/data/Generation/environment/envs/faceswap/lib/python3.9/site-packages/keras/engine/base_layer.py", line 1096, in __call__
outputs = call_fn(inputs, *args, **kwargs)
File "/data/Generation/environment/envs/faceswap/lib/python3.9/site-packages/keras/utils/traceback_utils.py", line 92, in error_handler
return fn(*args, **kwargs)
File "/data/Generation/environment/envs/faceswap/lib/python3.9/site-packages/keras/layers/convolutional.py", line 248, in call
outputs = self.convolution_op(inputs, self.kernel)
File "/data/Generation/environment/envs/faceswap/lib/python3.9/site-packages/keras/layers/convolutional.py", line 233, in convolution_op
return tf.nn.convolution(
Node: 'original/encoder/conv_128_1_conv2d/Conv2D'
DNN library is not found.
[[{{node original/encoder/conv_128_1_conv2d/Conv2D}}]] [Op:__inference_train_function_7661]
06/04/2023 10:23:52 MainProcess MainThread launcher execute_script CRITICAL An unexpected crash has occurred. Crash report written to '/data/Generation/faceswap/crash_report.2023.06.04.102347805853.log'. You MUST provide this file if seeking assistance. Please verify you are running the latest version of faceswap before reporting
Here is the crash report I get during extract phase.
Code: Select all
06/04/2023 10:25:27 MainProcess MainThread logger log_setup INFO Log level set to: INFO
06/04/2023 10:25:30 MainProcess MainThread plugin_loader _import INFO Loading Detect from S3Fd plugin...
06/04/2023 10:25:30 MainProcess MainThread plugin_loader _import INFO Loading Align from Fan plugin...
06/04/2023 10:25:30 MainProcess MainThread plugin_loader _import INFO Loading Mask from Components plugin...
06/04/2023 10:25:30 MainProcess MainThread plugin_loader _import INFO Loading Mask from Extended plugin...
06/04/2023 10:25:30 MainProcess MainThread extract process INFO Starting, this may take a while...
06/04/2023 10:25:30 MainProcess MainThread extract __init__ INFO Output Directory: /data/Generation/faceswap/faces/02/002
06/04/2023 10:25:30 MainProcess MainThread _base initialize INFO Initializing S3FD (Detect)...
06/04/2023 10:25:32 MainProcess MainThread _base initialize INFO Initialized S3FD (Detect) with batchsize of 1
06/04/2023 10:25:32 MainProcess MainThread _base initialize INFO Initializing FAN (Align)...
06/04/2023 10:25:54 MainProcess MainThread launcher execute_script ERROR Got Exception on main handler:
Traceback (most recent call last):
File "/data/Generation/faceswap/lib/cli/launcher.py", line 230, in execute_script
process.process()
File "/data/Generation/faceswap/scripts/extract.py", line 178, in process
extract.process()
File "/data/Generation/faceswap/scripts/extract.py", line 702, in process
self._run_extraction()
File "/data/Generation/faceswap/scripts/extract.py", line 722, in _run_extraction
self._extractor.launch()
File "/data/Generation/faceswap/plugins/extract/pipeline.py", line 271, in launch
self._launch_plugin(phase)
File "/data/Generation/faceswap/plugins/extract/pipeline.py", line 700, in _launch_plugin
plugin.initialize(**kwargs)
File "/data/Generation/faceswap/plugins/extract/align/_base/aligner.py", line 199, in initialize
super().initialize(*args, **kwargs)
File "/data/Generation/faceswap/plugins/extract/_base.py", line 481, in initialize
self.init_model()
File "/data/Generation/faceswap/plugins/extract/align/fan.py", line 50, in init_model
self.model.predict(placeholder)
File "/data/Generation/faceswap/lib/model/session.py", line 105, in predict
return self._model.predict(feed, verbose=0, batch_size=batch_size)
File "/data/Generation/environment/envs/faceswap/lib/python3.9/site-packages/keras/utils/traceback_utils.py", line 67, in error_handler
raise e.with_traceback(filtered_tb) from None
File "/data/Generation/environment/envs/faceswap/lib/python3.9/site-packages/tensorflow/python/eager/execute.py", line 54, in quick_execute
tensors = pywrap_tfe.TFE_Py_Execute(ctx._handle, device_name, op_name,
tensorflow.python.framework.errors_impl.UnimplementedError: Graph execution error:
Detected at node 'model_1/conv1/Conv2D' defined at (most recent call last):
File "/data/Generation/faceswap/faceswap.py", line 57, in <module>
_main()
File "/data/Generation/faceswap/faceswap.py", line 53, in _main
arguments.func(arguments)
File "/data/Generation/faceswap/lib/cli/launcher.py", line 230, in execute_script
process.process()
File "/data/Generation/faceswap/scripts/extract.py", line 178, in process
extract.process()
File "/data/Generation/faceswap/scripts/extract.py", line 702, in process
self._run_extraction()
File "/data/Generation/faceswap/scripts/extract.py", line 722, in _run_extraction
self._extractor.launch()
File "/data/Generation/faceswap/plugins/extract/pipeline.py", line 271, in launch
self._launch_plugin(phase)
File "/data/Generation/faceswap/plugins/extract/pipeline.py", line 700, in _launch_plugin
plugin.initialize(**kwargs)
File "/data/Generation/faceswap/plugins/extract/align/_base/aligner.py", line 199, in initialize
super().initialize(*args, **kwargs)
File "/data/Generation/faceswap/plugins/extract/_base.py", line 481, in initialize
self.init_model()
File "/data/Generation/faceswap/plugins/extract/align/fan.py", line 50, in init_model
self.model.predict(placeholder)
File "/data/Generation/faceswap/lib/model/session.py", line 105, in predict
return self._model.predict(feed, verbose=0, batch_size=batch_size)
File "/data/Generation/environment/envs/faceswap/lib/python3.9/site-packages/keras/utils/traceback_utils.py", line 64, in error_handler
return fn(*args, **kwargs)
File "/data/Generation/environment/envs/faceswap/lib/python3.9/site-packages/keras/engine/training.py", line 1982, in predict
tmp_batch_outputs = self.predict_function(iterator)
File "/data/Generation/environment/envs/faceswap/lib/python3.9/site-packages/keras/engine/training.py", line 1801, in predict_function
return step_function(self, iterator)
File "/data/Generation/environment/envs/faceswap/lib/python3.9/site-packages/keras/engine/training.py", line 1790, in step_function
outputs = model.distribute_strategy.run(run_step, args=(data,))
File "/data/Generation/environment/envs/faceswap/lib/python3.9/site-packages/keras/engine/training.py", line 1783, in run_step
outputs = model.predict_step(data)
File "/data/Generation/environment/envs/faceswap/lib/python3.9/site-packages/keras/engine/training.py", line 1751, in predict_step
return self(x, training=False)
File "/data/Generation/environment/envs/faceswap/lib/python3.9/site-packages/keras/utils/traceback_utils.py", line 64, in error_handler
return fn(*args, **kwargs)
File "/data/Generation/environment/envs/faceswap/lib/python3.9/site-packages/keras/engine/base_layer.py", line 1096, in __call__
outputs = call_fn(inputs, *args, **kwargs)
File "/data/Generation/environment/envs/faceswap/lib/python3.9/site-packages/keras/utils/traceback_utils.py", line 92, in error_handler
return fn(*args, **kwargs)
File "/data/Generation/environment/envs/faceswap/lib/python3.9/site-packages/keras/engine/functional.py", line 451, in call
return self._run_internal_graph(
File "/data/Generation/environment/envs/faceswap/lib/python3.9/site-packages/keras/engine/functional.py", line 589, in _run_internal_graph
outputs = node.layer(*args, **kwargs)
File "/data/Generation/environment/envs/faceswap/lib/python3.9/site-packages/keras/utils/traceback_utils.py", line 64, in error_handler
return fn(*args, **kwargs)
File "/data/Generation/environment/envs/faceswap/lib/python3.9/site-packages/keras/engine/base_layer.py", line 1096, in __call__
outputs = call_fn(inputs, *args, **kwargs)
File "/data/Generation/environment/envs/faceswap/lib/python3.9/site-packages/keras/utils/traceback_utils.py", line 92, in error_handler
return fn(*args, **kwargs)
File "/data/Generation/environment/envs/faceswap/lib/python3.9/site-packages/keras/layers/convolutional.py", line 248, in call
outputs = self.convolution_op(inputs, self.kernel)
File "/data/Generation/environment/envs/faceswap/lib/python3.9/site-packages/keras/layers/convolutional.py", line 233, in convolution_op
return tf.nn.convolution(
Node: 'model_1/conv1/Conv2D'
DNN library is not found.
[[{{node model_1/conv1/Conv2D}}]] [Op:__inference_predict_function_17968]
06/04/2023 10:25:54 MainProcess MainThread launcher execute_script CRITICAL An unexpected crash has occurred. Crash report written to '/data/Generation/faceswap/crash_report.2023.06.04.102549152473.log'. You MUST provide this file if seeking assistance. Please verify you are running the latest version of faceswap before reporting
It is worth noting that when I use mtcnn for detection and cv2-dnn for alignment, the extraction process works normally. That is, use the following command
Code: Select all
python faceswap.py extract -i /***/01.mp4 -o ./faces/02 -A cv2-dnn -D mtcnn
I'm using an Nvidia 3080Ti graphics card and guaranteed to have 11G of video memory at runtime.
I tried some solutions and still can't solve this problem, hope to get some help.