Training Error for my Phaze-A model on a recent Faceswap update

If training is failing to start, and you are not receiving an error message telling you what to do, tell us about it here


Forum rules

Read the FAQs and search the forum before posting a new topic.

This forum is for reporting errors with the Training process. If you want to get tips, or better understand the Training process, then you should look in the Training Discussion forum.

Please mark any answers that fixed your problems so others can find the solutions.

Locked
User avatar
Johnny99
Posts: 9
Joined: Sat Apr 04, 2020 8:03 pm
Has thanked: 2 times
Been thanked: 1 time

Training Error for my Phaze-A model on a recent Faceswap update

Post by Johnny99 »

I'm getting this error for my Phaze-A model when I click on the train button. Other models for me are okay. It was working fine before updating today - the last time I updated before this was probably a month ago. Here's the error:

08/26/2022 13:41:18 INFO Loading Trainer from Original plugin...
08/26/2022 13:41:19 CRITICAL Error caught! Exiting...
08/26/2022 13:41:19 ERROR Caught exception in thread: 'training'
08/26/2022 13:41:25 ERROR Got Exception on main handler:
Traceback (most recent call last):
File "M:\Faceswap_2-0\FaceSwap2\lib\cli\launcher.py", line 201, in execute_script
process.process()
File "M:\Faceswap_2-0\FaceSwap2\scripts\train.py", line 216, in process
self.
end_thread(thread, err)
File "M:\Faceswap_2-0\FaceSwap2\scripts\train.py", line 256, in end_thread
thread.join()
File "M:\Faceswap_2-0\FaceSwap2\lib\multithreading.py", line 217, in join
raise thread.err[1].with_traceback(thread.err[2])
File "M:\Faceswap_2-0\FaceSwap2\lib\multithreading.py", line 96, in run
self.
target(*self.args, **self.kwargs)
File "M:\Faceswap_2-0\FaceSwap2\scripts\train.py", line 278, in _training
raise err
File "M:\Faceswap_2-0\FaceSwap2\scripts\train.py", line 268, in training
self.
run_training_cycle(model, trainer)
File "M:\Faceswap_2-0\FaceSwap2\scripts\train.py", line 353, in run_training_cycle
trainer.train_one_step(viewer, timelapse)
File "M:\Faceswap_2-0\FaceSwap2\plugins\train\trainer\_base.py", line 210, in train_one_step
loss: List[float] = self.
model.model.train_on_batch(model_inputs, y=model_targets)
File "C:\Users\Phuong\Anaconda3\envs\faceswap\lib\site-packages\keras\engine\training.py", line 2144, in train_on_batch
logs = self.train_function(iterator)
File "C:\Users\Phuong\Anaconda3\envs\faceswap\lib\site-packages\tensorflow\python\util\traceback_utils.py", line 153, in error_handler
raise e.with_traceback(filtered_tb) from None
File "C:\Users\Phuong\AppData\Local\Temp\__autograph_generated_file7hqun1br.py", line 15, in tftrain_function
retval_ = ag
.converted_call(ag.ld(step_function), (ag.ld(self), ag__.ld(iterator)), None, fscope)
File "C:\Users\Phuong\Anaconda3\envs\faceswap\lib\site-packages\keras\engine\training.py", line 1040, in step_function
outputs = model.distribute_strategy.run(run_step, args=(data,))
File "C:\Users\Phuong\Anaconda3\envs\faceswap\lib\site-packages\keras\engine\training.py", line 1030, in run_step
outputs = model.train_step(data)
File "C:\Users\Phuong\Anaconda3\envs\faceswap\lib\site-packages\keras\engine\training.py", line 889, in train_step
y_pred = self(x, training=True)
File "C:\Users\Phuong\Anaconda3\envs\faceswap\lib\site-packages\keras\utils\traceback_utils.py", line 67, in error_handler
raise e.with_traceback(filtered_tb) from None
File "C:\Users\Phuong\Anaconda3\envs\faceswap\lib\site-packages\keras\engine\input_spec.py", line 264, in assert_input_compatibility
raise ValueError(f'Input {input_index} of layer "{layer_name}" is '
ValueError: in user code:

File "C:\Users\Phuong\Anaconda3\envs\faceswap\lib\site-packages\keras\engine\training.py", line 1051, in train_function *
return step_function(self, iterator)
File "C:\Users\Phuong\Anaconda3\envs\faceswap\lib\site-packages\keras\engine\training.py", line 1040, in step_function **
outputs = model.distribute_strategy.run(run_step, args=(data,))
File "C:\Users\Phuong\Anaconda3\envs\faceswap\lib\site-packages\keras\engine\training.py", line 1030, in run_step **
outputs = model.train_step(data)
File "C:\Users\Phuong\Anaconda3\envs\faceswap\lib\site-packages\keras\engine\training.py", line 889, in train_step
y_pred = self(x, training=True)
File "C:\Users\Phuong\Anaconda3\envs\faceswap\lib\site-packages\keras\utils\traceback_utils.py", line 67, in error_handler
raise e.with_traceback(filtered_tb) from None
File "C:\Users\Phuong\Anaconda3\envs\faceswap\lib\site-packages\keras\engine\input_spec.py", line 264, in assert_input_compatibility
raise ValueError(f'Input {input_index} of layer "{layer_name}" is '

ValueError: Input 0 of layer "phaze_a" is incompatible with the layer: expected shape=(None, 128, 128, 3), found shape=(7, 816, 816, 3)

User avatar
torzdf
Posts: 2649
Joined: Fri Jul 12, 2019 12:53 am
Answers: 159
Has thanked: 128 times
Been thanked: 622 times

Re: Training Error for my Phaze-A model on a recent Faceswap update

Post by torzdf »

I would need the full crash report

My word is final

User avatar
Johnny99
Posts: 9
Joined: Sat Apr 04, 2020 8:03 pm
Has thanked: 2 times
Been thanked: 1 time

Re: Training Error for my Phaze-A model on a recent Faceswap update

Post by Johnny99 »

I've pm'ed you the full crash report. Thank you.

User avatar
torzdf
Posts: 2649
Joined: Fri Jul 12, 2019 12:53 am
Answers: 159
Has thanked: 128 times
Been thanked: 622 times

Re: Training Error for my Phaze-A model on a recent Faceswap update

Post by torzdf »

Johnny99 wrote: Sat Aug 27, 2022 12:48 am

I've pm'ed you the full crash report. Thank you.

Ok, I have just tried training a model with the exact settings as in your provide crash report (all the settings were default by the look of it). It started up fine.

Are you using an existing model? If so, please could you provide the state.json file from your model folder.

What size are your extract images (shouldn't make a difference, but might as well rule it out).

My word is final

User avatar
Johnny99
Posts: 9
Joined: Sat Apr 04, 2020 8:03 pm
Has thanked: 2 times
Been thanked: 1 time

Re: Training Error for my Phaze-A model on a recent Faceswap update

Post by Johnny99 »

Hey @torzdf . I am using an existing model. I've been using this model for months now. I even tried a older backup of the model and tried a different folder of faceset that's also known to work. The size of the extracted images has also been 512x512. Thanks.

phaze_a_state.zip
state json
(4.67 KiB) Downloaded 105 times
User avatar
Johnny99
Posts: 9
Joined: Sat Apr 04, 2020 8:03 pm
Has thanked: 2 times
Been thanked: 1 time

Re: Training Error for my Phaze-A model on a recent Faceswap update

Post by Johnny99 »

When I created this model, it did use all default settings except for the output size which I made bigger than the default 128. I haven't tried training a new phaze-a model.

User avatar
torzdf
Posts: 2649
Joined: Fri Jul 12, 2019 12:53 am
Answers: 159
Has thanked: 128 times
Been thanked: 622 times

Re: Training Error for my Phaze-A model on a recent Faceswap update

Post by torzdf »

Thanks. Found bug and squashed.

Please update.

My word is final

User avatar
Johnny99
Posts: 9
Joined: Sat Apr 04, 2020 8:03 pm
Has thanked: 2 times
Been thanked: 1 time

Re: Training Error for my Phaze-A model on a recent Faceswap update

Post by Johnny99 »

Everything looks good on my side now. Thanks for the quick fix!

Locked