could not broadcast input array

If training is failing to start, and you are not receiving an error message telling you what to do, tell us about it here


Forum rules

Read the FAQs and search the forum before posting a new topic.

This forum is for reporting errors with the Training process. If you want to get tips, or better understand the Training process, then you should look in the Training Discussion forum.

Please mark any answers that fixed your problems so others can find the solutions.

Locked
User avatar
Antipope1
Posts: 35
Joined: Thu Jul 02, 2020 10:51 am
Has thanked: 6 times
Been thanked: 3 times

could not broadcast input array

Post by Antipope1 »

Good morning.

I started a new project last night, everything extracted as it should and training started normally, however 250(ish) iterations in I kept getting this error. Any ideas why?

Code: Select all

 1 
 
01/06/2021 01:33:49 MainProcess     _run_1                         aligned_face    extract_face                   DEBUG    _extract_face called without a loaded image. Returning empty face.
 2 
 
01/06/2021 01:33:49 MainProcess     _run_1                         training_data   _expand_partials               DEBUG    Generating mask. side: 'b', filename: 'C:\FaceSwap\InputB\Extracted\reex\04073_0.jpg'
 3 
 
01/06/2021 01:33:49 MainProcess     _run_1                         aligned_face    extract_face                   DEBUG    _extract_face called without a loaded image. Returning empty face.
 4 
 
01/06/2021 01:33:49 MainProcess     _run_1                         training_data   _expand_partials               DEBUG    Generating mask. side: 'b', filename: 'C:\FaceSwap\InputB\Extracted\reex\2e_014617_0_0.png'
 5 
 
01/06/2021 01:33:49 MainProcess     _run_1                         aligned_face    extract_face                   DEBUG    _extract_face called without a loaded image. Returning empty face.
 6 
 
01/06/2021 01:33:49 MainProcess     _run_1                         training_data   _expand_partials               DEBUG    Generating mask. side: 'b', filename: 'C:\FaceSwap\InputB\Extracted\reex\08588_0.jpg'
 7 
 
01/06/2021 01:33:49 MainProcess     _run_1                         aligned_face    extract_face                   DEBUG    _extract_face called without a loaded image. Returning empty face.
 8 
 
01/06/2021 01:33:49 MainProcess     _run_1                         training_data   _expand_partials               DEBUG    Generating mask. side: 'b', filename: 'C:\FaceSwap\InputB\Extracted\reex\06702_0.jpg'
 9 
 
01/06/2021 01:33:49 MainProcess     _run_1                         aligned_face    extract_face                   DEBUG    _extract_face called without a loaded image. Returning empty face.
 10 
 
01/06/2021 01:33:49 MainProcess     _run_1                         training_data   _expand_partials               DEBUG    Generating mask. side: 'b', filename: 'C:\FaceSwap\InputB\Extracted\reex\09749_0.png'
 11 
 
01/06/2021 01:33:49 MainProcess     _run_1                         aligned_face    extract_face                   DEBUG    _extract_face called without a loaded image. Returning empty face.
 12 
 
01/06/2021 01:33:49 MainProcess     _run_1                         training_data   _expand_partials               DEBUG    Generating mask. side: 'b', filename: 'C:\FaceSwap\InputB\Extracted\reex\2e_017563_0_1.png'
 13 
 
01/06/2021 01:33:49 MainProcess     _run_1                         aligned_face    extract_face                   DEBUG    _extract_face called without a loaded image. Returning empty face.
 14 
 
01/06/2021 01:33:49 MainProcess     _run_1                         training_data   _expand_partials               DEBUG    Generating mask. side: 'b', filename: 'C:\FaceSwap\InputB\Extracted\reex\01003 (2)_0.png'
 15 
 
01/06/2021 01:33:49 MainProcess     _run_1                         aligned_face    extract_face                   DEBUG    _extract_face called without a loaded image. Returning empty face.
 16 
 
01/06/2021 01:33:49 MainProcess     _run_1                         training_data   _expand_partials               DEBUG    Generating mask. side: 'b', filename: 'C:\FaceSwap\InputB\Extracted\reex\06192_0.png'
 17 
 
01/06/2021 01:33:49 MainProcess     _run_1                         aligned_face    extract_face                   DEBUG    _extract_face called without a loaded image. Returning empty face.
 18 
 
01/06/2021 01:33:49 MainProcess     _run_1                         training_data   _expand_partials               DEBUG    Generating mask. side: 'b', filename: 'C:\FaceSwap\InputB\Extracted\reex\00355_0.png'
 19 
 
01/06/2021 01:33:49 MainProcess     _run_1                         aligned_face    extract_face                   DEBUG    _extract_face called without a loaded image. Returning empty face.
 20 
 
01/06/2021 01:33:49 MainProcess     _run_1                         training_data   _expand_partials               DEBUG    Generating mask. side: 'b', filename: 'C:\FaceSwap\InputB\Extracted\reex\2e_004573_0_1.png'
 21 
 
01/06/2021 01:33:49 MainProcess     _run_1                         aligned_face    extract_face                   DEBUG    _extract_face called without a loaded image. Returning empty face.
 22 
 
01/06/2021 01:33:49 MainProcess     _run_1                         training_data   _expand_partials               DEBUG    Generating mask. side: 'b', filename: 'C:\FaceSwap\InputB\Extracted\reex\00420 (2)_0.png'
 23 
 
01/06/2021 01:33:49 MainProcess     _run_1                         aligned_face    extract_face                   DEBUG    _extract_face called without a loaded image. Returning empty face.
 24 
 
01/06/2021 01:33:49 MainProcess     _run_1                         training_data   _expand_partials               DEBUG    Generating mask. side: 'b', filename: 'C:\FaceSwap\InputB\Extracted\reex\06399_0.png'
 25 
 
01/06/2021 01:33:49 MainProcess     _run_1                         aligned_face    extract_face                   DEBUG    _extract_face called without a loaded image. Returning empty face.
 26 
 
01/06/2021 01:33:49 MainProcess     _run_1                         training_data   _expand_partials               DEBUG    Generating mask. side: 'b', filename: 'C:\FaceSwap\InputB\Extracted\reex\04146_0.jpg'
 27 
 
01/06/2021 01:33:49 MainProcess     _run_1                         aligned_face    extract_face                   DEBUG    _extract_face called without a loaded image. Returning empty face.
 28 
 
01/06/2021 01:33:49 MainProcess     _run_1                         training_data   _expand_partials               DEBUG    Generating mask. side: 'b', filename: 'C:\FaceSwap\InputB\Extracted\reex\02328_0.png'
 29 
 
01/06/2021 01:33:49 MainProcess     _run_1                         aligned_face    extract_face                   DEBUG    _extract_face called without a loaded image. Returning empty face.
 30 
 
01/06/2021 01:33:49 MainProcess     _run_1                         training_data   _expand_partials               DEBUG    Generating mask. side: 'b', filename: 'C:\FaceSwap\InputB\Extracted\reex\09435_0.png'
 31 
 
01/06/2021 01:33:49 MainProcess     _run_1                         aligned_face    extract_face                   DEBUG    _extract_face called without a loaded image. Returning empty face.
 32 
 
01/06/2021 01:33:50 MainProcess     _run_1                         training_data   _expand_partials               DEBUG    Generating mask. side: 'b', filename: 'C:\FaceSwap\InputB\Extracted\reex\01260_1.png'
 33 
 
01/06/2021 01:33:50 MainProcess     _run_1                         aligned_face    extract_face                   DEBUG    _extract_face called without a loaded image. Returning empty face.
 34 
 
01/06/2021 01:33:50 MainProcess     _run_1                         training_data   _expand_partials               DEBUG    Generating mask. side: 'b', filename: 'C:\FaceSwap\InputB\Extracted\reex\09877_0.png'
 35 
 
01/06/2021 01:33:50 MainProcess     _run_1                         aligned_face    extract_face                   DEBUG    _extract_face called without a loaded image. Returning empty face.
 36 
 
01/06/2021 01:33:50 MainProcess     _run_1                         training_data   _expand_partials               DEBUG    Generating mask. side: 'b', filename: 'C:\FaceSwap\InputB\Extracted\reex\00067 (2)_0.png'
 37 
 
01/06/2021 01:33:50 MainProcess     _run_1                         aligned_face    extract_face                   DEBUG    _extract_face called without a loaded image. Returning empty face.
 38 
 
01/06/2021 01:33:50 MainProcess     _run_1                         training_data   _expand_partials               DEBUG    Generating mask. side: 'b', filename: 'C:\FaceSwap\InputB\Extracted\reex\03230_0.jpg'
 39 
 
01/06/2021 01:33:50 MainProcess     _run_1                         aligned_face    extract_face                   DEBUG    _extract_face called without a loaded image. Returning empty face.
 40 
 
01/06/2021 01:33:50 MainProcess     _run_1                         training_data   _expand_partials               DEBUG    Generating mask. side: 'b', filename: 'C:\FaceSwap\InputB\Extracted\reex\05604_0.png'
 41 
 
01/06/2021 01:33:50 MainProcess     _run_1                         aligned_face    extract_face                   DEBUG    _extract_face called without a loaded image. Returning empty face.
 42 
 
01/06/2021 01:33:50 MainProcess     _run_1                         training_data   _expand_partials               DEBUG    Generating mask. side: 'b', filename: 'C:\FaceSwap\InputB\Extracted\reex\07271_0.JPG'
 43 
 
01/06/2021 01:33:50 MainProcess     _run_1                         aligned_face    extract_face                   DEBUG    _extract_face called without a loaded image. Returning empty face.
 44 
 
01/06/2021 01:33:50 MainProcess     _run_1                         training_data   _expand_partials               DEBUG    Generating mask. side: 'b', filename: 'C:\FaceSwap\InputB\Extracted\reex\04569_0.png'
 45 
 
01/06/2021 01:33:50 MainProcess     _run_1                         aligned_face    extract_face                   DEBUG    _extract_face called without a loaded image. Returning empty face.
 46 
 
01/06/2021 01:33:50 MainProcess     _run_1                         training_data   _expand_partials               DEBUG    Generating mask. side: 'b', filename: 'C:\FaceSwap\InputB\Extracted\reex\08511_0.png'
 47 
 
01/06/2021 01:33:50 MainProcess     _run_1                         aligned_face    extract_face                   DEBUG    _extract_face called without a loaded image. Returning empty face.
 48 
 
01/06/2021 01:33:50 MainProcess     _run_1                         training_data   _expand_partials               DEBUG    Generating mask. side: 'b', filename: 'C:\FaceSwap\InputB\Extracted\reex\10749_0.png'
 49 
 
01/06/2021 01:33:50 MainProcess     _run_1                         aligned_face    extract_face                   DEBUG    _extract_face called without a loaded image. Returning empty face.
 50 
 
01/06/2021 01:33:50 MainProcess     _run_1                         training_data   _expand_partials               DEBUG    Generating mask. side: 'b', filename: 'C:\FaceSwap\InputB\Extracted\reex\2e_012355_0_0.png'
 51 
 
01/06/2021 01:33:50 MainProcess     _run_1                         aligned_face    extract_face                   DEBUG    _extract_face called without a loaded image. Returning empty face.
 52 
 
01/06/2021 01:33:50 MainProcess     _run_1                         training_data   _expand_partials               DEBUG    Generating mask. side: 'b', filename: 'C:\FaceSwap\InputB\Extracted\reex\02575_0.png'
 53 
 
01/06/2021 01:33:50 MainProcess     _run_1                         aligned_face    extract_face                   DEBUG    _extract_face called without a loaded image. Returning empty face.
 54 
 
01/06/2021 01:33:50 MainProcess     _run_1                         training_data   _expand_partials               DEBUG    Generating mask. side: 'b', filename: 'C:\FaceSwap\InputB\Extracted\reex\00255 (3)_0.png'
 55 
 
01/06/2021 01:33:50 MainProcess     _run_1                         aligned_face    extract_face                   DEBUG    _extract_face called without a loaded image. Returning empty face.
 56 
 
01/06/2021 01:33:50 MainProcess     _run_1                         training_data   _expand_partials               DEBUG    Generating mask. side: 'b', filename: 'C:\FaceSwap\InputB\Extracted\reex\01223 (2)_0.png'
 57 
 
01/06/2021 01:33:50 MainProcess     _run_1                         aligned_face    extract_face                   DEBUG    _extract_face called without a loaded image. Returning empty face.
 58 
 
01/06/2021 01:33:50 MainProcess     _run_1                         training_data   _expand_partials               DEBUG    Generating mask. side: 'b', filename: 'C:\FaceSwap\InputB\Extracted\reex\04146_0.jpg'
 59 
 
01/06/2021 01:33:50 MainProcess     _run_1                         aligned_face    extract_face                   DEBUG    _extract_face called without a loaded image. Returning empty face.
 60 
 
01/06/2021 01:33:50 MainProcess     _run_1                         training_data   _expand_partials               DEBUG    Generating mask. side: 'b', filename: 'C:\FaceSwap\InputB\Extracted\reex\02328_0.png'
 61 
 
01/06/2021 01:33:50 MainProcess     _run_1                         aligned_face    extract_face                   DEBUG    _extract_face called without a loaded image. Returning empty face.
 62 
 
01/06/2021 01:33:50 MainProcess     _run_1                         training_data   _expand_partials               DEBUG    Generating mask. side: 'b', filename: 'C:\FaceSwap\InputB\Extracted\reex\09435_0.png'
 63 
 
01/06/2021 01:33:50 MainProcess     _run_1                         aligned_face    extract_face                   DEBUG    _extract_face called without a loaded image. Returning empty face.
 64 
 
01/06/2021 01:33:50 MainProcess     _run_1                         training_data   _expand_partials               DEBUG    Generating mask. side: 'b', filename: 'C:\FaceSwap\InputB\Extracted\reex\01260_1.png'
 65 
 
01/06/2021 01:33:50 MainProcess     _run_1                         aligned_face    extract_face                   DEBUG    _extract_face called without a loaded image. Returning empty face.
 66 
 
01/06/2021 01:33:50 MainProcess     _run_1                         training_data   _expand_partials               DEBUG    Generating mask. side: 'b', filename: 'C:\FaceSwap\InputB\Extracted\reex\09877_0.png'
 67 
 
01/06/2021 01:33:50 MainProcess     _run_1                         aligned_face    extract_face                   DEBUG    _extract_face called without a loaded image. Returning empty face.
 68 
 
01/06/2021 01:33:50 MainProcess     _run_1                         training_data   _expand_partials               DEBUG    Generating mask. side: 'b', filename: 'C:\FaceSwap\InputB\Extracted\reex\00067 (2)_0.png'
 69 
 
01/06/2021 01:33:50 MainProcess     _run_1                         aligned_face    extract_face                   DEBUG    _extract_face called without a loaded image. Returning empty face.
 70 
 
01/06/2021 01:33:50 MainProcess     _run_1                         training_data   _expand_partials               DEBUG    Generating mask. side: 'b', filename: 'C:\FaceSwap\InputB\Extracted\reex\03230_0.jpg'
 71 
 
01/06/2021 01:33:50 MainProcess     _run_1                         aligned_face    extract_face                   DEBUG    _extract_face called without a loaded image. Returning empty face.
 72 
 
01/06/2021 01:33:50 MainProcess     _run_1                         training_data   _expand_partials               DEBUG    Generating mask. side: 'b', filename: 'C:\FaceSwap\InputB\Extracted\reex\05604_0.png'
 73 
 
01/06/2021 01:33:50 MainProcess     _run_1                         aligned_face    extract_face                   DEBUG    _extract_face called without a loaded image. Returning empty face.
 74 
 
01/06/2021 01:33:50 MainProcess     _run_1                         training_data   _expand_partials               DEBUG    Generating mask. side: 'b', filename: 'C:\FaceSwap\InputB\Extracted\reex\07271_0.JPG'
 75 
 
01/06/2021 01:33:50 MainProcess     _run_1                         aligned_face    extract_face                   DEBUG    _extract_face called without a loaded image. Returning empty face.
 76 
 
01/06/2021 01:33:50 MainProcess     _run_1                         training_data   _expand_partials               DEBUG    Generating mask. side: 'b', filename: 'C:\FaceSwap\InputB\Extracted\reex\04569_0.png'
 77 
 
01/06/2021 01:33:50 MainProcess     _run_1                         aligned_face    extract_face                   DEBUG    _extract_face called without a loaded image. Returning empty face.
 78 
 
01/06/2021 01:33:50 MainProcess     _run_1                         training_data   _expand_partials               DEBUG    Generating mask. side: 'b', filename: 'C:\FaceSwap\InputB\Extracted\reex\08511_0.png'
 79 
 
01/06/2021 01:33:50 MainProcess     _run_1                         aligned_face    extract_face                   DEBUG    _extract_face called without a loaded image. Returning empty face.
 80 
 
01/06/2021 01:33:50 MainProcess     _run_1                         training_data   _expand_partials               DEBUG    Generating mask. side: 'b', filename: 'C:\FaceSwap\InputB\Extracted\reex\10749_0.png'
 81 
 
01/06/2021 01:33:50 MainProcess     _run_1                         aligned_face    extract_face                   DEBUG    _extract_face called without a loaded image. Returning empty face.
 82 
 
01/06/2021 01:33:50 MainProcess     _run_1                         training_data   _expand_partials               DEBUG    Generating mask. side: 'b', filename: 'C:\FaceSwap\InputB\Extracted\reex\2e_012355_0_0.png'
 83 
 
01/06/2021 01:33:50 MainProcess     _run_1                         aligned_face    extract_face                   DEBUG    _extract_face called without a loaded image. Returning empty face.
 84 
 
01/06/2021 01:33:50 MainProcess     _run_1                         training_data   _expand_partials               DEBUG    Generating mask. side: 'b', filename: 'C:\FaceSwap\InputB\Extracted\reex\02575_0.png'
 85 
 
01/06/2021 01:33:50 MainProcess     _run_1                         aligned_face    extract_face                   DEBUG    _extract_face called without a loaded image. Returning empty face.
 86 
 
01/06/2021 01:33:50 MainProcess     _run_1                         training_data   _expand_partials               DEBUG    Generating mask. side: 'b', filename: 'C:\FaceSwap\InputB\Extracted\reex\00255 (3)_0.png'
 87 
 
01/06/2021 01:33:50 MainProcess     _run_1                         aligned_face    extract_face                   DEBUG    _extract_face called without a loaded image. Returning empty face.
 88 
 
01/06/2021 01:33:50 MainProcess     _run_1                         training_data   _expand_partials               DEBUG    Generating mask. side: 'b', filename: 'C:\FaceSwap\InputB\Extracted\reex\01223 (2)_0.png'
 89 
 
01/06/2021 01:33:50 MainProcess     _run_1                         aligned_face    extract_face                   DEBUG    _extract_face called without a loaded image. Returning empty face.
 90 
 
01/06/2021 01:33:51 MainProcess     _run_1                         multithreading  run                            DEBUG    Error in thread (_run_1): could not broadcast input array from shape (89,314,3) into shape (89,0,3)
 91 
 
01/06/2021 01:33:52 MainProcess     _training_0                    multithreading  check_and_raise_error          DEBUG    Thread error caught: [(<class 'ValueError'>, ValueError('could not broadcast input array from shape (89,314,3) into shape (89,0,3)'), <traceback object at 0x000001F9832E2280>)]
 92 
 
01/06/2021 01:33:52 MainProcess     _training_0                    multithreading  run                            DEBUG    Error in thread (_training_0): could not broadcast input array from shape (89,314,3) into shape (89,0,3)
 93 
 
01/06/2021 01:33:52 MainProcess     _run_0                         multithreading  run                            DEBUG    Error in thread (_run_0): could not broadcast input array from shape (89,314,3) into shape (89,0,3)
 94 
 
01/06/2021 01:33:53 MainProcess     MainThread                     train           _monitor                       DEBUG    Thread error detected
 95 
 
01/06/2021 01:33:53 MainProcess     MainThread                     train           _monitor                       DEBUG    Closed Monitor
 96 
 
01/06/2021 01:33:53 MainProcess     MainThread                     train           _end_thread                    DEBUG    Ending Training thread
 97 
 
01/06/2021 01:33:53 MainProcess     MainThread                     train           _end_thread                    CRITICAL Error caught! Exiting...
 98 
 
01/06/2021 01:33:53 MainProcess     MainThread                     multithreading  join                           DEBUG    Joining Threads: '_training'
 99 
 
01/06/2021 01:33:53 MainProcess     MainThread                     multithreading  join                           DEBUG    Joining Thread: '_training_0'
 100 
 
01/06/2021 01:33:53 MainProcess     MainThread                     multithreading  join                           ERROR    Caught exception in thread: '_training_0'
 101 
 
Traceback (most recent call last):
 102 
 
  File "C:\Users\Matty\faceswap\lib\cli\launcher.py", line 182, in execute_script
 103 
 
process.process()
 104 
 
  File "C:\Users\Matty\faceswap\scripts\train.py", line 170, in process
 105 
 
self._end_thread(thread, err)
 106 
 
  File "C:\Users\Matty\faceswap\scripts\train.py", line 210, in _end_thread
 107 
 
thread.join()
 108 
 
  File "C:\Users\Matty\faceswap\lib\multithreading.py", line 121, in join
 109 
 
raise thread.err[1].with_traceback(thread.err[2])
 110 
 
  File "C:\Users\Matty\faceswap\lib\multithreading.py", line 37, in run
 111 
 
self._target(*self._args, **self._kwargs)
 112 
 
  File "C:\Users\Matty\faceswap\scripts\train.py", line 232, in _training
 113 
 
raise err
 114 
 
  File "C:\Users\Matty\faceswap\scripts\train.py", line 222, in _training
 115 
 
self._run_training_cycle(model, trainer)
 116 
 
  File "C:\Users\Matty\faceswap\scripts\train.py", line 302, in _run_training_cycle
 117 
 
trainer.train_one_step(viewer, timelapse)
 118 
 
  File "C:\Users\Matty\faceswap\plugins\train\trainer\_base.py", line 236, in train_one_step
 119 
 
model_inputs, model_targets = self._feeder.get_batch()
 120 
 
  File "C:\Users\Matty\faceswap\plugins\train\trainer\_base.py", line 450, in get_batch
 121 
 
batch = next(self._feeds[side])
 122 
 
  File "C:\Users\Matty\faceswap\lib\multithreading.py", line 156, in iterator
 123 
 
self.check_and_raise_error()
 124 
 
  File "C:\Users\Matty\faceswap\lib\multithreading.py", line 84, in check_and_raise_error
 125 
 
raise error[1].with_traceback(error[2])
 126 
 
  File "C:\Users\Matty\faceswap\lib\multithreading.py", line 37, in run
 127 
 
self._target(*self._args, **self._kwargs)
 128 
 
  File "C:\Users\Matty\faceswap\lib\multithreading.py", line 145, in _run
 129 
 
for item in self.generator(*self._gen_args, **self._gen_kwargs):
 130 
 
  File "C:\Users\Matty\faceswap\lib\training_data.py", line 206, in _minibatch
 131 
 
yield self._process_batch(img_paths, side)
 132 
 
  File "C:\Users\Matty\faceswap\lib\training_data.py", line 215, in _process_batch
 133 
 
batch, landmarks = self._crop_to_center(filenames, batch, side)
 134 
 
  File "C:\Users\Matty\faceswap\lib\training_data.py", line 319, in _crop_to_center
 135 
 
cropped = np.array([align.extract_face(img) for align, img in zip(aligned, batch)])
 136 
 
  File "C:\Users\Matty\faceswap\lib\training_data.py", line 319, in <listcomp>
 137 
 
cropped = np.array([align.extract_face(img) for align, img in zip(aligned, batch)])
 138 
 
  File "C:\Users\Matty\faceswap\lib\align\aligned_face.py", line 341, in extract_face
 139 
 
image = self._convert_centering(image)
 140 
 
  File "C:\Users\Matty\faceswap\lib\align\aligned_face.py", line 380, in _convert_centering
 141 
 
out[slices["out"][0], slices["out"][1], :] = image[slices["in"][0], slices["in"][1], :]
 142 
 
ValueError: could not broadcast input array from shape (89,314,3) into shape (89,0,3)
 143 
 
 144 
 
============ System Information ============
 145 
 
encoding:            cp1252
 146 
 
git_branch:          master
 147 
 
git_commits:         b1420a6 Merge pull request #1096 from deepfakes/dev-tf2.4. 29667b4 Expand support for tf2.2-2.4. 3438198 Merge branch 'master' into dev-tf2.4. b67f91e travis test typo fix. 199f6ca update travis test
 148 
 
gpu_cuda:            No global version found. Check Conda packages for Conda Cuda
 149 
 
gpu_cudnn:           No global version found. Check Conda packages for Conda cuDNN
 150 
 
gpu_devices:         GPU_0: GeForce RTX 2060 SUPER
 151 
 
gpu_devices_active:  GPU_0
 152 
 
gpu_driver:          457.51
 153 
 
gpu_vram:            GPU_0: 8192MB
 154 
 
os_machine:          AMD64
 155 
 
os_platform:         Windows-10-10.0.19041-SP0
 156 
 
os_release:          10
 157 
 
py_command:          C:\Users\Matty\faceswap\faceswap.py train -A C:/FaceSwap/InputA/ex/mb1st/1 -ala C:/FaceSwap/InputA/vid/mb1st/alignments_MERGED_20210106_003612.fsa -B C:/FaceSwap/InputB/Extracted/reex -alb C:/FaceSwap/InputB/Extracted/alignments.fsa -m C:/FaceSwap/Models/sinew model_snapshot_150000_iters -t realface -bs 16 -it 1000000 -s 250 -ss 25000 -ps 100 -L INFO -gui
 158 
 
py_conda_version:    conda 4.9.1
 159 
 
py_implementation:   CPython
 160 
 
py_version:          3.8.5
 161 
 
py_virtual_env:      True
 162 
 
sys_cores:           12
 163 
 
sys_processor:       AMD64 Family 25 Model 33 Stepping 0, AuthenticAMD
 164 
 
sys_ram:             Total: 32689MB, Available: 25463MB, Used: 7226MB, Free: 25463MB
 165 
 
 166 
 
=============== Pip Packages ===============
 167 
 
absl-py==0.11.0
 168 
 
astunparse==1.6.3
 169 
 
cachetools==4.1.1
 170 
 
certifi==2020.6.20
 171 
 
chardet==3.0.4
 172 
 
cycler==0.10.0
 173 
 
fastcluster==1.1.26
 174 
 
ffmpy==0.2.3
 175 
 
gast==0.3.3
 176 
 
google-auth==1.23.0
 177 
 
google-auth-oauthlib==0.4.2
 178 
 
google-pasta==0.2.0
 179 
 
grpcio==1.33.2
 180 
 
h5py==2.10.0
 181 
 
idna==2.10
 182 
 
imageio @ file:///tmp/build/80754af9/imageio_1594161405741/work
 183 
 
imageio-ffmpeg @ file:///home/conda/feedstock_root/build_artifacts/imageio-ffmpeg_1589202782679/work
 184 
 
joblib @ file:///tmp/build/80754af9/joblib_1601912903842/work
 185 
 
Keras-Preprocessing==1.1.2
 186 
 
kiwisolver @ file:///C:/ci/kiwisolver_1604014703538/work
 187 
 
Markdown==3.3.3
 188 
 
matplotlib @ file:///C:/ci/matplotlib-base_1592837548929/work
 189 
 
mkl-fft==1.2.0
 190 
 
mkl-random==1.1.1
 191 
 
mkl-service==2.3.0
 192 
 
numpy==1.18.5
 193 
 
nvidia-ml-py3 @ git+https://github.com/deepfakes/nvidia-ml-py3.git@6fc29ac84b32bad877f078cb4a777c1548a00bf6
 194 
 
oauthlib==3.1.0
 195 
 
olefile==0.46
 196 
 
opencv-python==4.4.0.46
 197 
 
opt-einsum==3.3.0
 198 
 
pathlib==1.0.1
 199 
 
Pillow @ file:///C:/ci/pillow_1603823068645/work
 200 
 
protobuf==3.13.0
 201 
 
psutil @ file:///C:/ci/psutil_1598370330503/work
 202 
 
pyasn1==0.4.8
 203 
 
pyasn1-modules==0.2.8
 204 
 
pyparsing==2.4.7
 205 
 
python-dateutil==2.8.1
 206 
 
pywin32==227
 207 
 
requests==2.24.0
 208 
 
requests-oauthlib==1.3.0
 209 
 
rsa==4.6
 210 
 
scikit-learn @ file:///C:/ci/scikit-learn_1598377018496/work
 211 
 
scipy @ file:///C:/ci/scipy_1592916963468/work
 212 
 
sip==4.19.13
 213 
 
six==1.15.0
 214 
 
tensorboard==2.2.2
 215 
 
tensorboard-plugin-wit==1.7.0
 216 
 
tensorflow-gpu==2.2.1
 217 
 
tensorflow-gpu-estimator==2.2.0
 218 
 
termcolor==1.1.0
 219 
 
threadpoolctl @ file:///tmp/tmp9twdgx9k/threadpoolctl-2.1.0-py3-none-any.whl
 220 
 
tornado==6.0.4
 221 
 
tqdm @ file:///tmp/build/80754af9/tqdm_1602185206534/work
 222 
 
urllib3==1.25.11
 223 
 
Werkzeug==1.0.1
 224 
 
wincertstore==0.2
 225 
 
wrapt==1.12.1
 226 
 
 227 
 
============== Conda Packages ==============
 228 
 
# packages in environment at C:\Users\Matty\MiniConda3\envs\faceswap:
 229 
 
#
 230 
 
# Name                    Version                   Build  Channel
 231 
 
absl-py                   0.11.0                   pypi_0    pypi
 232 
 
astunparse                1.6.3                    pypi_0    pypi
 233 
 
blas                      1.0                         mkl  
234 ca-certificates 2020.10.14 0
235 cachetools 4.1.1 pypi_0 pypi 236 certifi 2020.6.20 pyhd3eb1b0_3
237 chardet 3.0.4 pypi_0 pypi 238 cudatoolkit 10.1.243 h74a9793_0
239 cudnn 7.6.5 cuda10.1_0
240 cycler 0.10.0 py38_0
241 fastcluster 1.1.26 py38h251f6bf_2 conda-forge 242 ffmpeg 4.3.1 ha925a31_0 conda-forge 243 ffmpy 0.2.3 pypi_0 pypi 244 freetype 2.10.4 hd328e21_0
245 gast 0.3.3 pypi_0 pypi 246 git 2.23.0 h6bb4b03_0
247 google-auth 1.23.0 pypi_0 pypi 248 google-auth-oauthlib 0.4.2 pypi_0 pypi 249 google-pasta 0.2.0 pypi_0 pypi 250 grpcio 1.33.2 pypi_0 pypi 251 h5py 2.10.0 pypi_0 pypi 252 icc_rt 2019.0.0 h0cc432a_1
253 icu 58.2 ha925a31_3
254 idna 2.10 pypi_0 pypi 255 imageio 2.9.0 py_0
256 imageio-ffmpeg 0.4.2 py_0 conda-forge 257 intel-openmp 2020.2 254
258 joblib 0.17.0 py_0
259 jpeg 9b hb83a4c4_2
260 keras-preprocessing 1.1.2 pypi_0 pypi 261 kiwisolver 1.3.0 py38hd77b12b_0
262 libpng 1.6.37 h2a8f88b_0
263 libtiff 4.1.0 h56a325e_1
264 lz4-c 1.9.2 hf4a77e7_3
265 markdown 3.3.3 pypi_0 pypi 266 matplotlib 3.2.2 0
267 matplotlib-base 3.2.2 py38h64f37c6_0
268 mkl 2020.2 256
269 mkl-service 2.3.0 py38hb782905_0
270 mkl_fft 1.2.0 py38h45dec08_0
271 mkl_random 1.1.1 py38h47e9c7a_0
272 numpy 1.18.5 pypi_0 pypi 273 nvidia-ml-py3 7.352.1 pypi_0 pypi 274 oauthlib 3.1.0 pypi_0 pypi 275 olefile 0.46 py_0
276 opencv-python 4.4.0.46 pypi_0 pypi 277 openssl 1.1.1h he774522_0
278 opt-einsum 3.3.0 pypi_0 pypi 279 pathlib 1.0.1 py_1
280 pillow 8.0.1 py38h4fa10fc_0
281 pip 20.2.4 py38haa95532_0
282 protobuf 3.13.0 pypi_0 pypi 283 psutil 5.7.2 py38he774522_0
284 pyasn1 0.4.8 pypi_0 pypi 285 pyasn1-modules 0.2.8 pypi_0 pypi 286 pyparsing 2.4.7 py_0
287 pyqt 5.9.2 py38ha925a31_4
288 python 3.8.5 h5fd99cc_1
289 python-dateutil 2.8.1 py_0
290 python_abi 3.8 1_cp38 conda-forge 291 pywin32 227 py38he774522_1
292 qt 5.9.7 vc14h73c81de_0
293 requests 2.24.0 pypi_0 pypi 294 requests-oauthlib 1.3.0 pypi_0 pypi 295 rsa 4.6 pypi_0 pypi 296 scikit-learn 0.23.2 py38h47e9c7a_0
297 scipy 1.5.0 py38h9439919_0
298 setuptools 50.3.0 py38haa95532_1
299 sip 4.19.13 py38ha925a31_0
300 six 1.15.0 py_0
301 sqlite 3.33.0 h2a8f88b_0
302 tensorboard 2.2.2 pypi_0 pypi 303 tensorboard-plugin-wit 1.7.0 pypi_0 pypi 304 tensorflow-gpu 2.2.1 pypi_0 pypi 305 tensorflow-gpu-estimator 2.2.0 pypi_0 pypi 306 termcolor 1.1.0 pypi_0 pypi 307 threadpoolctl 2.1.0 pyh5ca1d4c_0
308 tk 8.6.10 he774522_0
309 tornado 6.0.4 py38he774522_1
310 tqdm 4.50.2 py_0
311 urllib3 1.25.11 pypi_0 pypi 312 vc 14.1 h0510ff6_4
313 vs2015_runtime 14.16.27012 hf0eaf9b_3
314 werkzeug 1.0.1 pypi_0 pypi 315 wheel 0.35.1 py_0
316 wincertstore 0.2 py38_0
317 wrapt 1.12.1 pypi_0 pypi 318 xz 5.2.5 h62dcd97_0
319 zlib 1.2.11 h62dcd97_4
320 zstd 1.4.5 h04227a9_0
321 322 =============== State File ================= 323 { 324 "name": "realface", 325 "sessions": { 326 "1": { 327 "timestamp": 1609895068.9486148, 328 "no_logs": false, 329 "loss_names": [ 330 "total", 331 "face_a", 332 "face_b" 333 ], 334 "batchsize": 16, 335 "iterations": 290, 336 "config": { 337 "learning_rate": 5e-05, 338 "allow_growth": false, 339 "convert_batchsize": 16, 340 "eye_multiplier": 3, 341 "mouth_multiplier": 2 342 } 343 }, 344 "2": { 345 "timestamp": 1609895356.13302, 346 "no_logs": false, 347 "loss_names": [ 348 "total", 349 "face_a", 350 "face_b" 351 ], 352 "batchsize": 16, 353 "iterations": 500, 354 "config": { 355 "learning_rate": 5e-05, 356 "allow_growth": false, 357 "convert_batchsize": 16, 358 "eye_multiplier": 3, 359 "mouth_multiplier": 2 360 } 361 }, 362 "3": { 363 "timestamp": 1609896290.6571288, 364 "no_logs": false, 365 "loss_names": [ 366 "total", 367 "face_a", 368 "face_b" 369 ], 370 "batchsize": 16, 371 "iterations": 1000, 372 "config": { 373 "learning_rate": 5e-05, 374 "allow_growth": false, 375 "convert_batchsize": 16, 376 "eye_multiplier": 3, 377 "mouth_multiplier": 2 378 } 379 } 380 }, 381 "lowest_avg_loss": { 382 "a": 0.07222099900245667, 383 "b": 0.06717865192890167 384 }, 385 "iterations": 1790, 386 "config": { 387 "centering": "face", 388 "coverage": 68.75, 389 "optimizer": "adam", 390 "learning_rate": 5e-05, 391 "allow_growth": false, 392 "mixed_precision": true, 393 "convert_batchsize": 16, 394 "loss_function": "ssim", 395 "mask_loss_function": "mse", 396 "l2_reg_term": 100, 397 "eye_multiplier": 3, 398 "mouth_multiplier": 2, 399 "penalized_mask_loss": true, 400 "mask_type": "extended", 401 "mask_blur_kernel": 3, 402 "mask_threshold": 4, 403 "learn_mask": false, 404 "input_size": 96, 405 "output_size": 160, 406 "dense_nodes": 1536, 407 "complexity_encoder": 128, 408 "complexity_decoder": 512 409 } 410 } 411 412 ================= Configs ================== 413 --------- .faceswap --------- 414 backend: nvidia 415 416 --------- convert.ini --------- 417 418 [color.color_transfer] 419 clip: True 420 preserve_paper: True 421 422 [color.manual_balance] 423 colorspace: HSV 424 balance_1: 0.0 425 balance_2: 0.0 426 balance_3: 0.0 427 contrast: 0.0 428 brightness: 0.0 429 430 [color.match_hist] 431 threshold: 99.0 432 433 [mask.box_blend] 434 type: gaussian 435 distance: 11.0 436 radius: 5.0 437 passes: 1 438 439 [mask.mask_blend] 440 type: normalized 441 kernel_size: 3 442 passes: 4 443 threshold: 4 444 erosion: 0.0 445 446 [scaling.sharpen] 447 method: none 448 amount: 150 449 radius: 0.3 450 threshold: 5.0 451 452 [writer.ffmpeg] 453 container: mp4 454 codec: libx264 455 crf: 23 456 preset: medium 457 tune: none 458 profile: auto 459 level: auto 460 skip_mux: False 461 462 [writer.gif] 463 fps: 25 464 loop: 0 465 palettesize: 256 466 subrectangles: False 467 468 [writer.opencv] 469 format: png 470 draw_transparent: False 471 jpg_quality: 75 472 png_compress_level: 3 473 474 [writer.pillow] 475 format: png 476 draw_transparent: False 477 optimize: False 478 gif_interlace: True 479 jpg_quality: 75 480 png_compress_level: 3 481 tif_compression: tiff_deflate 482 483 --------- extract.ini --------- 484 485 [global] 486 allow_growth: False 487 488 [align.fan] 489 batch-size: 12 490 491 [detect.cv2_dnn] 492 confidence: 50 493 494 [detect.mtcnn] 495 minsize: 20 496 threshold_1: 0.6 497 threshold_2: 0.7 498 threshold_3: 0.7 499 scalefactor: 0.709 500 batch-size: 8 501 502 [detect.s3fd] 503 confidence: 70 504 batch-size: 4 505 506 [mask.unet_dfl] 507 batch-size: 8 508 509 [mask.vgg_clear] 510 batch-size: 6 511 512 [mask.vgg_obstructed] 513 batch-size: 2 514 515 --------- gui.ini --------- 516 517 [global] 518 fullscreen: True 519 tab: extract 520 options_panel_width: 30 521 console_panel_height: 20 522 icon_size: 14 523 font: default 524 font_size: 9 525 autosave_last_session: prompt 526 timeout: 120 527 auto_load_model_stats: True 528 529 --------- train.ini --------- 530 531 [global] 532 centering: face 533 coverage: 68.75 534 icnr_init: False 535 conv_aware_init: True 536 optimizer: adam 537 learning_rate: 5e-05 538 reflect_padding: True 539 allow_growth: False 540 mixed_precision: True 541 convert_batchsize: 16 542 543 [global.loss] 544 loss_function: ssim 545 mask_loss_function: mse 546 l2_reg_term: 100 547 eye_multiplier: 3 548 mouth_multiplier: 2 549 penalized_mask_loss: True 550 mask_type: extended 551 mask_blur_kernel: 3 552 mask_threshold: 4 553 learn_mask: False 554 555 [model.dfaker] 556 output_size: 128 557 558 [model.dfl_h128] 559 lowmem: False 560 561 [model.dfl_sae] 562 input_size: 128 563 clipnorm: True 564 architecture: df 565 autoencoder_dims: 0 566 encoder_dims: 42 567 decoder_dims: 21 568 multiscale_decoder: False 569 570 [model.dlight] 571 features: best 572 details: good 573 output_size: 256 574 575 [model.original] 576 lowmem: False 577 578 [model.realface] 579 input_size: 96 580 output_size: 160 581 dense_nodes: 1536 582 complexity_encoder: 128 583 complexity_decoder: 512 584 585 [model.unbalanced] 586 input_size: 128 587 lowmem: False 588 clipnorm: True 589 nodes: 1024 590 complexity_encoder: 128 591 complexity_decoder_a: 384 592 complexity_decoder_b: 512 593 594 [model.villain] 595 lowmem: False 596 597 [trainer.original] 598 preview_images: 14 599 zoom_amount: 5 600 rotation_range: 10 601 shift_range: 5 602 flip_chance: 50 603 disable_warp: False 604 color_lightness: 30 605 color_ab: 8 606 color_clahe_chance: 50 607 color_clahe_max_size: 4 608
The beginning of the end.
User avatar
torzdf
Posts: 2670
Joined: Fri Jul 12, 2019 12:53 am
Answers: 159
Has thanked: 131 times
Been thanked: 625 times

Re: could not broadcast input array

Post by torzdf »

It looks like there is a problem with a specific image. Unfortunately the output doesn't give me enough information on why this is....

If you could run with Loglevel set to Trace and then provide the full faceswap.log when it fails, we may be able to diagnose.

Note: Trace logging will slow down the process and generate a massive log file, so only do it in this instance.

My word is final

Locked