KeyError when training with mask and timelapse
Posted: Sat Jun 20, 2020 10:15 pm
Hello 
I'm having this error while I start training with mask vgg-clear and timelapse.
I think the problem is that it can't find a mask for timelapse images, because training without using the mask and training without using timelapse both work.
I extracted faces from A and B with the masker, then I copied 14 images from faces folders to two new folders that I use for timelapse, but this happens when I start training. Moreover if I point timelapse input folder to A and B main faces folder it works. Am I doing something wrong?
Code: Select all
06/20/2020 23:34:55 MainProcess _training_0 multithreading __init__ DEBUG Initialized BackgroundGenerator: '_run'
06/20/2020 23:34:55 MainProcess _training_0 multithreading start DEBUG Starting thread(s): '_run'
06/20/2020 23:34:55 MainProcess _training_0 multithreading start DEBUG Starting thread 1 of 2: '_run_0'
06/20/2020 23:34:55 MainProcess _run_0 training_data _minibatch DEBUG Loading minibatch generator: (image_count: 14, side: 'b', do_shuffle: False)
06/20/2020 23:34:55 MainProcess _training_0 multithreading start DEBUG Starting thread 2 of 2: '_run_1'
06/20/2020 23:34:55 MainProcess _run_1 training_data _minibatch DEBUG Loading minibatch generator: (image_count: 14, side: 'b', do_shuffle: False)
06/20/2020 23:34:55 MainProcess _training_0 multithreading start DEBUG Started all threads '_run': 2
06/20/2020 23:34:55 MainProcess _training_0 _base set_timelapse_feed DEBUG Set time-lapse feed
06/20/2020 23:34:55 MainProcess _training_0 _base _setup DEBUG Set up time-lapse
06/20/2020 23:34:55 MainProcess _run_1 multithreading run DEBUG Error in thread (_run_1): '/home/user/Files/A/Timelapse/A01.png'
06/20/2020 23:34:55 MainProcess _run_0 multithreading run DEBUG Error in thread (_run_0): '/home/user/Files/B/Timelapse/B01.png'
06/20/2020 23:34:55 MainProcess _training_0 multithreading check_and_raise_error DEBUG Thread error caught: [(<class 'KeyError'>, KeyError('/home/user/Files/A/Timelapse/A01.png'), <traceback object at 0x7f237c7b2f50>)]
06/20/2020 23:34:55 MainProcess _training_0 multithreading run DEBUG Error in thread (_training_0): '/home/user/Files/A/Timelapse/A01.png'
06/20/2020 23:34:55 MainProcess _run_0 multithreading run DEBUG Error in thread (_run_0): '/home/user/Files/A/Timelapse/A01.png'
06/20/2020 23:34:56 MainProcess MainThread train _monitor DEBUG Thread error detected
06/20/2020 23:34:56 MainProcess MainThread train _monitor DEBUG Closed Monitor
06/20/2020 23:34:56 MainProcess MainThread train _end_thread DEBUG Ending Training thread
06/20/2020 23:34:56 MainProcess MainThread train _end_thread CRITICAL Error caught! Exiting...
06/20/2020 23:34:56 MainProcess MainThread multithreading join DEBUG Joining Threads: '_training'
06/20/2020 23:34:56 MainProcess MainThread multithreading join DEBUG Joining Thread: '_training_0'
06/20/2020 23:34:56 MainProcess MainThread multithreading join ERROR Caught exception in thread: '_training_0'
Traceback (most recent call last):
File "/home/user/faceswap/lib/cli/launcher.py", line 155, in execute_script
process.process()
File "/home/user/faceswap/scripts/train.py", line 161, in process
self._end_thread(thread, err)
File "/home/user/faceswap/scripts/train.py", line 201, in _end_thread
thread.join()
File "/home/user/faceswap/lib/multithreading.py", line 121, in join
raise thread.err[1].with_traceback(thread.err[2])
File "/home/user/faceswap/lib/multithreading.py", line 37, in run
self._target(*self._args, **self._kwargs)
File "/home/user/faceswap/scripts/train.py", line 226, in _training
raise err
File "/home/user/faceswap/scripts/train.py", line 216, in _training
self._run_training_cycle(model, trainer)
File "/home/user/faceswap/scripts/train.py", line 305, in _run_training_cycle
trainer.train_one_step(viewer, timelapse)
File "/home/user/faceswap/plugins/train/trainer/_base.py", line 316, in train_one_step
raise err
File "/home/user/faceswap/plugins/train/trainer/_base.py", line 290, in train_one_step
self._timelapse.get_sample(side, timelapse_kwargs)
File "/home/user/faceswap/plugins/train/trainer/_base.py", line 947, in get_sample
self._samples.images[side] = self._batchers[side].compile_timelapse_sample()
File "/home/user/faceswap/plugins/train/trainer/_base.py", line 539, in compile_timelapse_sample
batch = next(self._timelapse_feed)
File "/home/user/faceswap/lib/multithreading.py", line 156, in iterator
self.check_and_raise_error()
File "/home/user/faceswap/lib/multithreading.py", line 84, in check_and_raise_error
raise error[1].with_traceback(error[2])
File "/home/user/faceswap/lib/multithreading.py", line 37, in run
self._target(*self._args, **self._kwargs)
File "/home/user/faceswap/lib/multithreading.py", line 145, in _run
for item in self.generator(*self._gen_args, **self._gen_kwargs):
File "/home/user/faceswap/lib/training_data.py", line 189, in _minibatch
yield self._process_batch(img_paths, side)
File "/home/user/faceswap/lib/training_data.py", line 198, in _process_batch
batch = self._apply_mask(filenames, batch, side)
File "/home/user/faceswap/lib/training_data.py", line 253, in _apply_mask
for filename, face in zip(filenames, batch)], dtype=batch.dtype)
File "/home/user/faceswap/lib/training_data.py", line 253, in <listcomp>
for filename, face in zip(filenames, batch)], dtype=batch.dtype)
KeyError: '/home/user/Files/A/Timelapse/A01.png'