Page 1 of 1

Crash after running for a few minutes

Posted: Sun Aug 09, 2020 7:57 am
by tjzf

No matter how it is set, it will crash after running for a few minutes. This time, the return code is 1

Code: Select all

08/09/2020 15:47:22 MainProcess     _training_0     _base           _resize_sample            DEBUG    Resized sample: (side: 'b' shape: (14, 224, 224, 3))
08/09/2020 15:47:22 MainProcess     _training_0     _base           _overlay_foreground       DEBUG    Overlayed foreground. Shape: (14, 256, 256, 3)
08/09/2020 15:47:22 MainProcess     _training_0     _base           _overlay_foreground       DEBUG    Overlayed foreground. Shape: (14, 256, 256, 3)
08/09/2020 15:47:22 MainProcess     _training_0     _base           _overlay_foreground       DEBUG    Overlayed foreground. Shape: (14, 256, 256, 3)
08/09/2020 15:47:22 MainProcess     _training_0     _base           _resize_sample            DEBUG    Resizing sample: (side: 'b', sample.shape: (14, 256, 256, 3), target_size: 128, scale: 0.5)
08/09/2020 15:47:22 MainProcess     _training_0     _base           _resize_sample            DEBUG    Resized sample: (side: 'b' shape: (14, 128, 128, 3))
08/09/2020 15:47:22 MainProcess     _training_0     _base           _resize_sample            DEBUG    Resizing sample: (side: 'b', sample.shape: (14, 256, 256, 3), target_size: 128, scale: 0.5)
08/09/2020 15:47:22 MainProcess     _training_0     _base           _resize_sample            DEBUG    Resized sample: (side: 'b' shape: (14, 128, 128, 3))
08/09/2020 15:47:22 MainProcess     _training_0     _base           _resize_sample            DEBUG    Resizing sample: (side: 'b', sample.shape: (14, 256, 256, 3), target_size: 128, scale: 0.5)
08/09/2020 15:47:22 MainProcess     _training_0     _base           _resize_sample            DEBUG    Resized sample: (side: 'b' shape: (14, 128, 128, 3))
08/09/2020 15:47:22 MainProcess     _training_0     _base           _get_headers              DEBUG    side: 'b', width: 128
08/09/2020 15:47:22 MainProcess     _training_0     _base           _get_headers              DEBUG    height: 32, total_width: 384
08/09/2020 15:47:22 MainProcess     _training_0     _base           _get_headers              DEBUG    texts: ['Swap (B)', 'Swap > Swap', 'Swap > Original'], text_sizes: [(59, 9), (87, 9), (102, 9)], text_x: [34, 148, 269], text_y: 20
08/09/2020 15:47:22 MainProcess     _training_0     _base           _get_headers              DEBUG    header_box.shape: (32, 384, 3)
08/09/2020 15:47:22 MainProcess     _training_0     _base           _duplicate_headers        DEBUG    side: a header.shape: (32, 384, 3)
08/09/2020 15:47:22 MainProcess     _training_0     _base           _duplicate_headers        DEBUG    side: b header.shape: (32, 384, 3)
08/09/2020 15:47:22 MainProcess     _training_0     _base           _stack_images             DEBUG    Stack images
08/09/2020 15:47:22 MainProcess     _training_0     _base           get_transpose_axes        DEBUG    Even number of images to stack
08/09/2020 15:47:22 MainProcess     _training_0     _base           _stack_images             DEBUG    Stacked images
08/09/2020 15:47:22 MainProcess     _training_0     _base           show_sample               DEBUG    Compiled sample
08/09/2020 15:47:22 MainProcess     _training_0     _base           output_timelapse          DEBUG    Ouputting time-lapse
08/09/2020 15:47:22 MainProcess     _training_0     _base           show_sample               DEBUG    Showing sample
08/09/2020 15:47:22 MainProcess     _training_0     _base           _get_predictions          DEBUG    Getting Predictions
08/09/2020 15:47:22 MainProcess     _training_0     _base           _get_predictions          DEBUG    Returning predictions: {'a_a': (14, 64, 64, 3), 'b_a': (14, 64, 64, 3), 'a_b': (14, 64, 64, 3), 'b_b': (14, 64, 64, 3)}
08/09/2020 15:47:22 MainProcess     _training_0     _base           _to_full_frame            DEBUG    side: 'a', number of sample arrays: 3, prediction.shapes: [(14, 64, 64, 3), (14, 64, 64, 3)])
08/09/2020 15:47:22 MainProcess     _training_0     _base           _frame_overlay            DEBUG    full_size: 256, target_size: 224, color: (0, 0, 255)
08/09/2020 15:47:22 MainProcess     _training_0     _base           _frame_overlay            DEBUG    Overlayed background. Shape: (14, 256, 256, 3)
08/09/2020 15:47:22 MainProcess     _training_0     _base           _resize_sample            DEBUG    Resizing sample: (side: 'a', sample.shape: (14, 64, 64, 3), target_size: 224, scale: 3.5)
08/09/2020 15:47:22 MainProcess     _training_0     _base           _resize_sample            DEBUG    Resized sample: (side: 'a' shape: (14, 224, 224, 3))
08/09/2020 15:47:22 MainProcess     _training_0     _base           _resize_sample            DEBUG    Resizing sample: (side: 'a', sample.shape: (14, 64, 64, 3), target_size: 224, scale: 3.5)
08/09/2020 15:47:22 MainProcess     _training_0     _base           _resize_sample            DEBUG    Resized sample: (side: 'a' shape: (14, 224, 224, 3))
08/09/2020 15:47:22 MainProcess     _training_0     _base           _resize_sample            DEBUG    Resizing sample: (side: 'a', sample.shape: (14, 64, 64, 3), target_size: 224, scale: 3.5)
08/09/2020 15:47:22 MainProcess     _training_0     _base           _resize_sample            DEBUG    Resized sample: (side: 'a' shape: (14, 224, 224, 3))
08/09/2020 15:47:22 MainProcess     _training_0     _base           _overlay_foreground       DEBUG    Overlayed foreground. Shape: (14, 256, 256, 3)
08/09/2020 15:47:22 MainProcess     _training_0     _base           _overlay_foreground       DEBUG    Overlayed foreground. Shape: (14, 256, 256, 3)
08/09/2020 15:47:22 MainProcess     _training_0     _base           _overlay_foreground       DEBUG    Overlayed foreground. Shape: (14, 256, 256, 3)
08/09/2020 15:47:22 MainProcess     _training_0     _base           _get_headers              DEBUG    side: 'a', width: 256
08/09/2020 15:47:22 MainProcess     _training_0     _base           _get_headers              DEBUG    height: 64, total_width: 768
08/09/2020 15:47:22 MainProcess     _training_0     _base           _get_headers              DEBUG    texts: ['Original (A)', 'Original > Original', 'Original > Swap'], text_sizes: [(143, 18), (231, 18), (203, 18)], text_x: [56, 268, 538], text_y: 41
08/09/2020 15:47:22 MainProcess     _training_0     _base           _get_headers              DEBUG    header_box.shape: (64, 768, 3)
08/09/2020 15:47:22 MainProcess     _training_0     _base           _to_full_frame            DEBUG    side: 'b', number of sample arrays: 3, prediction.shapes: [(14, 64, 64, 3), (14, 64, 64, 3)])
08/09/2020 15:47:22 MainProcess     _training_0     _base           _frame_overlay            DEBUG    full_size: 256, target_size: 224, color: (0, 0, 255)
08/09/2020 15:47:22 MainProcess     _training_0     _base           _frame_overlay            DEBUG    Overlayed background. Shape: (14, 256, 256, 3)
08/09/2020 15:47:22 MainProcess     _training_0     _base           _resize_sample            DEBUG    Resizing sample: (side: 'b', sample.shape: (14, 64, 64, 3), target_size: 224, scale: 3.5)
08/09/2020 15:47:22 MainProcess     _training_0     _base           _resize_sample            DEBUG    Resized sample: (side: 'b' shape: (14, 224, 224, 3))
08/09/2020 15:47:22 MainProcess     _training_0     _base           _resize_sample            DEBUG    Resizing sample: (side: 'b', sample.shape: (14, 64, 64, 3), target_size: 224, scale: 3.5)
08/09/2020 15:47:22 MainProcess     _training_0     _base           _resize_sample            DEBUG    Resized sample: (side: 'b' shape: (14, 224, 224, 3))
08/09/2020 15:47:22 MainProcess     _training_0     _base           _resize_sample            DEBUG    Resizing sample: (side: 'b', sample.shape: (14, 64, 64, 3), target_size: 224, scale: 3.5)
08/09/2020 15:47:22 MainProcess     _training_0     _base           _resize_sample            DEBUG    Resized sample: (side: 'b' shape: (14, 224, 224, 3))
08/09/2020 15:47:22 MainProcess     _training_0     _base           _overlay_foreground       DEBUG    Overlayed foreground. Shape: (14, 256, 256, 3)
08/09/2020 15:47:22 MainProcess     _training_0     _base           _overlay_foreground       DEBUG    Overlayed foreground. Shape: (14, 256, 256, 3)
08/09/2020 15:47:22 MainProcess     _training_0     _base           _overlay_foreground       DEBUG    Overlayed foreground. Shape: (14, 256, 256, 3)
08/09/2020 15:47:22 MainProcess     _training_0     _base           _get_headers              DEBUG    side: 'b', width: 256
08/09/2020 15:47:22 MainProcess     _training_0     _base           _get_headers              DEBUG    height: 64, total_width: 768
08/09/2020 15:47:22 MainProcess     _training_0     _base           _get_headers              DEBUG    texts: ['Swap (B)', 'Swap > Swap', 'Swap > Original'], text_sizes: [(117, 18), (174, 18), (203, 18)], text_x: [69, 297, 538], text_y: 41
08/09/2020 15:47:22 MainProcess     _training_0     _base           _get_headers              DEBUG    header_box.shape: (64, 768, 3)
08/09/2020 15:47:22 MainProcess     _training_0     _base           _duplicate_headers        DEBUG    side: a header.shape: (64, 768, 3)
08/09/2020 15:47:22 MainProcess     _training_0     _base           _duplicate_headers        DEBUG    side: b header.shape: (64, 768, 3)
08/09/2020 15:47:22 MainProcess     _training_0     _base           _stack_images             DEBUG    Stack images
08/09/2020 15:47:22 MainProcess     _training_0     _base           get_transpose_axes        DEBUG    Even number of images to stack
08/09/2020 15:47:22 MainProcess     _training_0     _base           _stack_images             DEBUG    Stacked images
08/09/2020 15:47:22 MainProcess     _training_0     _base           show_sample               DEBUG    Compiled sample
08/09/2020 15:47:23 MainProcess     _training_0     _base           output_timelapse          DEBUG    Created time-lapse: 'Z:\Desktop\AIhuanlian\4\1596959242.jpg'
08/09/2020 15:47:23 MainProcess     _training_0     _base           save_models               DEBUG    Backing up and saving models
08/09/2020 15:47:23 MainProcess     _training_0     _base           get_save_averages         DEBUG    Getting save averages
08/09/2020 15:47:23 MainProcess     _training_0     _base           get_save_averages         DEBUG    Average losses since last save: {'a': 0.15506478816270827, 'b': 0.12676110863685608}
08/09/2020 15:47:23 MainProcess     _training_0     _base           check_loss_drop           DEBUG    Loss for 'a' has dropped
08/09/2020 15:47:23 MainProcess     _training_0     _base           check_loss_drop           DEBUG    Loss for 'b' has dropped
08/09/2020 15:47:23 MainProcess     _training_0     _base           should_backup             DEBUG    Lowest historical save iteration loss average: {'a': 0.20144721038639546, 'b': 0.16185383029282094}
08/09/2020 15:47:23 MainProcess     _training_0     _base           should_backup             DEBUG    Updating lowest save iteration average for 'a': 0.15506478816270827
08/09/2020 15:47:23 MainProcess     _training_0     _base           should_backup             DEBUG    Updating lowest save iteration average for 'b': 0.12676110863685608
08/09/2020 15:47:23 MainProcess     _training_0     _base           should_backup             DEBUG    Backing up: True
08/09/2020 15:47:23 MainProcess     _training_0     _base           save_models               INFO     Backing up models...
08/09/2020 15:47:23 MainProcess     ThreadPoolExecutor-434_0 backup_restore  backup_model              VERBOSE  Backing up: 'Z:\Desktop\AIhuanlian\moxingmulu\lightweight_decoder_A.h5' to 'Z:\Desktop\AIhuanlian\moxingmulu\lightweight_decoder_A.h5.bk'
08/09/2020 15:47:23 MainProcess     ThreadPoolExecutor-434_1 backup_restore  backup_model              VERBOSE  Backing up: 'Z:\Desktop\AIhuanlian\moxingmulu\lightweight_decoder_B.h5' to 'Z:\Desktop\AIhuanlian\moxingmulu\lightweight_decoder_B.h5.bk'
08/09/2020 15:47:23 MainProcess     ThreadPoolExecutor-434_2 backup_restore  backup_model              VERBOSE  Backing up: 'Z:\Desktop\AIhuanlian\moxingmulu\lightweight_encoder.h5' to 'Z:\Desktop\AIhuanlian\moxingmulu\lightweight_encoder.h5.bk'
08/09/2020 15:47:23 MainProcess     ThreadPoolExecutor-434_3 _base           save                      DEBUG    Saving State
08/09/2020 15:47:23 MainProcess     ThreadPoolExecutor-434_3 backup_restore  backup_model              VERBOSE  Backing up: 'Z:\Desktop\AIhuanlian\moxingmulu\lightweight_state.json' to 'Z:\Desktop\AIhuanlian\moxingmulu\lightweight_state.json.bk'
08/09/2020 15:47:23 MainProcess     ThreadPoolExecutor-434_0 _base           save                      DEBUG    Saving model: 'Z:\Desktop\AIhuanlian\moxingmulu\lightweight_decoder_A.h5'
08/09/2020 15:47:23 MainProcess     ThreadPoolExecutor-434_1 _base           save                      DEBUG    Saving model: 'Z:\Desktop\AIhuanlian\moxingmulu\lightweight_decoder_B.h5'
08/09/2020 15:47:23 MainProcess     ThreadPoolExecutor-434_2 _base           save                      DEBUG    Saving model: 'Z:\Desktop\AIhuanlian\moxingmulu\lightweight_encoder.h5'
08/09/2020 15:47:23 MainProcess     ThreadPoolExecutor-434_3 serializer      save                      DEBUG    filename: Z:\Desktop\AIhuanlian\moxingmulu\lightweight_state.json, data type: <class 'dict'>
08/09/2020 15:47:23 MainProcess     ThreadPoolExecutor-434_3 serializer      _check_extension          DEBUG    Original filename: 'Z:\Desktop\AIhuanlian\moxingmulu\lightweight_state.json', final filename: 'Z:\Desktop\AIhuanlian\moxingmulu\lightweight_state.json'
08/09/2020 15:47:23 MainProcess     ThreadPoolExecutor-434_3 serializer      marshal                   DEBUG    data type: <class 'dict'>
08/09/2020 15:47:23 MainProcess     ThreadPoolExecutor-434_3 serializer      marshal                   DEBUG    returned data type: <class 'bytes'>
08/09/2020 15:47:23 MainProcess     ThreadPoolExecutor-434_3 _base           save                      DEBUG    Saved State
08/09/2020 15:47:23 MainProcess     _training_0     _base           save_models               INFO     [Saved models] - Average since last save: face_loss_A: 0.15506, face_loss_B: 0.12676
08/09/2020 15:48:17 MainProcess     ThreadPoolExecutor-603_0 image           read_image                ERROR    Error while reading image. This is most likely caused by special characters in the filename: 'Z:\Desktop\AIhuanlian\xhssf\xhssf_000514_1 - 副本.png'
08/09/2020 15:48:18 MainProcess     ThreadPoolExecutor-605_0 image           read_image                ERROR    Error while reading image. This is most likely caused by special characters in the filename: 'Z:\Desktop\AIhuanlian\xhssf\xhssf_000514_1 - 副本.png'
08/09/2020 15:48:18 MainProcess     _run_0          multithreading  run                       DEBUG    Error in thread (_run_0): Error while reading image. This is most likely caused by special characters in the filename: 'Z:\Desktop\AIhuanlian\xhssf\xhssf_000514_1 - 副本.png'
08/09/2020 15:48:19 MainProcess     _training_0     multithreading  check_and_raise_error     DEBUG    Thread error caught: [(<class 'Exception'>, Exception("Error while reading image. This is most likely caused by special characters in the filename: 'Z:\\Desktop\\AIhuanlian\\xhssf\\xhssf_000514_1 - 副本.png'"), <traceback object at 0x0000019CD2ABD648>)]
08/09/2020 15:48:19 MainProcess     _training_0     multithreading  run                       DEBUG    Error in thread (_training_0): Error while reading image. This is most likely caused by special characters in the filename: 'Z:\Desktop\AIhuanlian\xhssf\xhssf_000514_1 - 副本.png'
08/09/2020 15:48:19 MainProcess     _run_1          multithreading  run                       DEBUG    Error in thread (_run_1): Error while reading image. This is most likely caused by special characters in the filename: 'Z:\Desktop\AIhuanlian\xhssf\xhssf_000514_1 - 副本.png'
08/09/2020 15:48:20 MainProcess     MainThread      train           _monitor                  DEBUG    Thread error detected
08/09/2020 15:48:20 MainProcess     MainThread      train           _monitor                  DEBUG    Closed Monitor
08/09/2020 15:48:20 MainProcess     MainThread      train           _end_thread               DEBUG    Ending Training thread
08/09/2020 15:48:20 MainProcess     MainThread      train           _end_thread               CRITICAL Error caught! Exiting...
08/09/2020 15:48:20 MainProcess     MainThread      multithreading  join                      DEBUG    Joining Threads: '_training'
08/09/2020 15:48:20 MainProcess     MainThread      multithreading  join                      DEBUG    Joining Thread: '_training_0'
08/09/2020 15:48:20 MainProcess     MainThread      multithreading  join                      ERROR    Caught exception in thread: '_training_0'
Traceback (most recent call last):
  File "C:\Users\15975\faceswap\lib\image.py", line 271, in read_image
    raise ValueError
ValueError

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "C:\Users\15975\faceswap\lib\cli\launcher.py", line 155, in execute_script
    process.process()
  File "C:\Users\15975\faceswap\scripts\train.py", line 161, in process
    self._end_thread(thread, err)
  File "C:\Users\15975\faceswap\scripts\train.py", line 201, in _end_thread
    thread.join()
  File "C:\Users\15975\faceswap\lib\multithreading.py", line 121, in join
    raise thread.err[1].with_traceback(thread.err[2])
  File "C:\Users\15975\faceswap\lib\multithreading.py", line 37, in run
    self._target(*self._args, **self._kwargs)
  File "C:\Users\15975\faceswap\scripts\train.py", line 226, in _training
    raise err
  File "C:\Users\15975\faceswap\scripts\train.py", line 216, in _training
    self._run_training_cycle(model, trainer)
  File "C:\Users\15975\faceswap\scripts\train.py", line 305, in _run_training_cycle
    trainer.train_one_step(viewer, timelapse)
  File "C:\Users\15975\faceswap\plugins\train\trainer\_base.py", line 316, in train_one_step
    raise err
  File "C:\Users\15975\faceswap\plugins\train\trainer\_base.py", line 283, in train_one_step
    loss[side] = batcher.train_one_batch()
  File "C:\Users\15975\faceswap\plugins\train\trainer\_base.py", line 422, in train_one_batch
    model_inputs, model_targets = self._get_next()
  File "C:\Users\15975\faceswap\plugins\train\trainer\_base.py", line 452, in _get_next
    batch = next(self._feed)
  File "C:\Users\15975\faceswap\lib\multithreading.py", line 156, in iterator
    self.check_and_raise_error()
  File "C:\Users\15975\faceswap\lib\multithreading.py", line 84, in check_and_raise_error
    raise error[1].with_traceback(error[2])
  File "C:\Users\15975\faceswap\lib\multithreading.py", line 37, in run
    self._target(*self._args, **self._kwargs)
  File "C:\Users\15975\faceswap\lib\multithreading.py", line 145, in _run
    for item in self.generator(*self._gen_args, **self._gen_kwargs):
  File "C:\Users\15975\faceswap\lib\training_data.py", line 189, in _minibatch
    yield self._process_batch(img_paths, side)
  File "C:\Users\15975\faceswap\lib\training_data.py", line 197, in _process_batch
    batch = read_image_batch(filenames)
  File "C:\Users\15975\faceswap\lib\image.py", line 334, in read_image_batch
    batch[return_indices[images[future]].pop()] = future.result()
  File "C:\Users\15975\MiniConda3\envs\faceswap\lib\concurrent\futures\_base.py", line 428, in result
    return self.__get_result()
  File "C:\Users\15975\MiniConda3\envs\faceswap\lib\concurrent\futures\_base.py", line 384, in __get_result
    raise self._exception
  File "C:\Users\15975\MiniConda3\envs\faceswap\lib\concurrent\futures\thread.py", line 57, in run
    result = self.fn(*self.args, **self.kwargs)
  File "C:\Users\15975\faceswap\lib\image.py", line 284, in read_image
    raise Exception(msg)
Exception: Error while reading image. This is most likely caused by special characters in the filename: 'Z:\Desktop\AIhuanlian\xhssf\xhssf_000514_1 - 副本.png'

============ System Information ============
encoding:            cp936
git_branch:          master
git_commits:         3fd26b5 Manual Tool (#1038)
gpu_cuda:            No global version found. Check Conda packages for Conda Cuda
gpu_cudnn:           No global version found. Check Conda packages for Conda cuDNN
gpu_devices:         GPU_0: GeForce GTX 1050 Ti
gpu_devices_active:  GPU_0
gpu_driver:          451.77
gpu_vram:            GPU_0: 4096MB
os_machine:          AMD64
os_platform:         Windows-10-10.0.18362-SP0
os_release:          10
py_command:          C:\Users\15975\faceswap\faceswap.py train -A Z:/Desktop/AIhuanlian/xhssf -B Z:/Desktop/AIhuanlian/wsczhengli -m Z:/Desktop/AIhuanlian/moxingmulu -t lightweight -bs 12 -it 1000000 -g 1 -msg -o -s 100 -ss 25000 -tia Z:/Desktop/AIhuanlian/xhssf -tib Z:/Desktop/AIhuanlian/wsczhengli -to Z:/Desktop/AIhuanlian/4 -ps 50 -ag -L INFO -gui
py_conda_version:    conda 4.8.3
py_implementation:   CPython
py_version:          3.7.7
py_virtual_env:      True
sys_cores:           12
sys_processor:       Intel64 Family 6 Model 158 Stepping 10, GenuineIntel
sys_ram:             Total: 24434MB, Available: 13908MB, Used: 10526MB, Free: 13908MB

=============== Pip Packages ===============
absl-py==0.9.0
astor==0.8.0
blinker==1.4
brotlipy==0.7.0
cachetools==4.1.0
certifi==2020.6.20
cffi==1.14.0
chardet==3.0.4
click==7.1.2
cloudpickle @ file:///tmp/build/80754af9/cloudpickle_1594141588948/work
cryptography==2.9.2
cycler==0.10.0
cytoolz==0.10.1
dask @ file:///tmp/build/80754af9/dask-core_1594156306305/work
decorator==4.4.2
fastcluster==1.1.26
ffmpy==0.2.3
gast==0.2.2
google-auth @ file:///tmp/build/80754af9/google-auth_1594357566944/work
google-auth-oauthlib==0.4.1
google-pasta==0.2.0
grpcio==1.27.2
h5py==2.10.0
idna @ file:///tmp/build/80754af9/idna_1593446292537/work
imageio @ file:///tmp/build/80754af9/imageio_1594161405741/work
imageio-ffmpeg @ file:///home/conda/feedstock_root/build_artifacts/imageio-ffmpeg_1589202782679/work
joblib @ file:///tmp/build/80754af9/joblib_1594236160679/work
Keras==2.2.4
Keras-Applications @ file:///tmp/build/80754af9/keras-applications_1594366238411/work
Keras-Preprocessing==1.1.0
kiwisolver==1.2.0
Markdown==3.1.1
matplotlib @ file:///C:/ci/matplotlib-base_1592846084747/work
mkl-fft==1.1.0
mkl-random==1.1.1
mkl-service==2.3.0
networkx @ file:///tmp/build/80754af9/networkx_1594377231366/work
numpy @ file:///C:/ci/numpy_and_numpy_base_1596233945180/work
nvidia-ml-py3 @ git+https://github.com/deepfakes/nvidia-ml-py3.git@6fc29ac84b32bad877f078cb4a777c1548a00bf6
oauthlib==3.1.0
olefile==0.46
opencv-python==4.3.0.36
opt-einsum==3.1.0
Pillow @ file:///C:/ci/pillow_1594298234712/work
protobuf==3.12.3
psutil==5.7.0
pyasn1==0.4.8
pyasn1-modules==0.2.7
pycparser @ file:///tmp/build/80754af9/pycparser_1594388511720/work
PyJWT==1.7.1
pyOpenSSL @ file:///tmp/build/80754af9/pyopenssl_1594392929924/work
pyparsing==2.4.7
pyreadline==2.1
PySocks @ file:///C:/ci/pysocks_1594394709107/work
python-dateutil==2.8.1
PyWavelets==1.1.1
pywin32==227
PyYAML==5.3.1
requests @ file:///tmp/build/80754af9/requests_1592841827918/work
requests-oauthlib==1.3.0
rsa==4.0
scikit-image==0.16.2
scikit-learn @ file:///C:/ci/scikit-learn_1592847564598/work
scipy @ file:///C:/ci/scipy_1592916958183/work
six==1.15.0
tensorboard==2.2.1
tensorboard-plugin-wit==1.6.0
tensorflow==1.15.0
tensorflow-estimator==1.15.1
termcolor==1.1.0
threadpoolctl @ file:///tmp/tmp9twdgx9k/threadpoolctl-2.1.0-py3-none-any.whl
toolz==0.10.0
toposort==1.5
tornado==6.0.4
tqdm @ file:///tmp/build/80754af9/tqdm_1593446365756/work
urllib3==1.25.9
Werkzeug==0.16.1
win-inet-pton==1.1.0
wincertstore==0.2
wrapt==1.12.1

============== Conda Packages ==============
# packages in environment at C:\Users\15975\MiniConda3\envs\faceswap:
#
# Name                    Version                   Build  Channel
_tflow_select             2.1.0                       gpu  
absl-py 0.9.0 py37_0
astor 0.8.0 py37_0
blas 1.0 mkl
blinker 1.4 py37_0
brotlipy 0.7.0 py37he774522_1000
ca-certificates 2020.6.24 0
cachetools 4.1.0 py_1
certifi 2020.6.20 py37_0
cffi 1.14.0 py37h7a1dbc1_0
chardet 3.0.4 py37_1003
click 7.1.2 py_0
cloudpickle 1.5.0 py_0
cryptography 2.9.2 py37h7a1dbc1_0
cudatoolkit 10.0.130 0
cudnn 7.6.5 cuda10.0_0
cycler 0.10.0 py37_0
cytoolz 0.10.1 py37he774522_0
dask-core 2.20.0 py_0
decorator 4.4.2 py_0
fastcluster 1.1.26 py37h9b59f54_1 conda-forge ffmpeg 4.3 ha925a31_1 conda-forge ffmpy 0.2.3 pypi_0 pypi freetype 2.10.2 hd328e21_0
gast 0.2.2 py37_0
git 2.23.0 h6bb4b03_0
google-auth 1.17.2 py_0
google-auth-oauthlib 0.4.1 py_2
google-pasta 0.2.0 py_0
grpcio 1.27.2 py37h351948d_0
h5py 2.10.0 py37h5e291fa_0
hdf5 1.10.4 h7ebc959_0
icc_rt 2019.0.0 h0cc432a_1
icu 58.2 ha925a31_3
idna 2.10 py_0
imageio 2.9.0 py_0
imageio-ffmpeg 0.4.2 py_0 conda-forge intel-openmp 2020.1 216
joblib 0.16.0 py_0
jpeg 9b hb83a4c4_2
keras 2.2.4 0
keras-applications 1.0.8 py_1
keras-base 2.2.4 py37_0
keras-preprocessing 1.1.0 py_1
kiwisolver 1.2.0 py37h74a9793_0
libpng 1.6.37 h2a8f88b_0
libprotobuf 3.12.3 h7bd577a_0
libtiff 4.1.0 h56a325e_1
lz4-c 1.9.2 h62dcd97_1
markdown 3.1.1 py37_0
matplotlib 3.2.2 0
matplotlib-base 3.2.2 py37h64f37c6_0
mkl 2020.1 216
mkl-service 2.3.0 py37hb782905_0
mkl_fft 1.1.0 py37h45dec08_0
mkl_random 1.1.1 py37h47e9c7a_0
networkx 2.4 py_1
numpy 1.19.1 py37h5510c5b_0
numpy-base 1.19.1 py37ha3acd2a_0
nvidia-ml-py3 7.352.1 pypi_0 pypi oauthlib 3.1.0 py_0
olefile 0.46 py37_0
opencv-python 4.3.0.36 pypi_0 pypi openssl 1.1.1g he774522_0
opt_einsum 3.1.0 py_0
pathlib 1.0.1 py37_2
pillow 7.2.0 py37hcc1f983_0
pip 20.1.1 py37_1
protobuf 3.12.3 py37h33f27b4_0
psutil 5.7.0 py37he774522_0
pyasn1 0.4.8 py_0
pyasn1-modules 0.2.7 py_0
pycparser 2.20 py_2
pyjwt 1.7.1 py37_0
pyopenssl 19.1.0 py_1
pyparsing 2.4.7 py_0
pyqt 5.9.2 py37h6538335_2
pyreadline 2.1 py37_1
pysocks 1.7.1 py37_1
python 3.7.7 h81c818b_4
python-dateutil 2.8.1 py_0
python_abi 3.7 1_cp37m conda-forge pywavelets 1.1.1 py37he774522_0
pywin32 227 py37he774522_1
pyyaml 5.3.1 py37he774522_1
qt 5.9.7 vc14h73c81de_0
requests 2.24.0 py_0
requests-oauthlib 1.3.0 py_0
rsa 4.0 py_0
scikit-image 0.16.2 py37h47e9c7a_0
scikit-learn 0.23.1 py37h25d0782_0
scipy 1.5.0 py37h9439919_0
setuptools 49.2.0 py37_0
sip 4.19.8 py37h6538335_0
six 1.15.0 py_0
sqlite 3.32.3 h2a8f88b_0
tensorboard 2.2.1 pyh532a8cf_0
tensorboard-plugin-wit 1.6.0 py_0
tensorflow 1.15.0 gpu_py37hc3743a6_0
tensorflow-base 1.15.0 gpu_py37h1afeea4_0
tensorflow-estimator 1.15.1 pyh2649769_0
tensorflow-gpu 1.15.0 h0d30ee6_0
termcolor 1.1.0 py37_1
threadpoolctl 2.1.0 pyh5ca1d4c_0
tk 8.6.10 he774522_0
toolz 0.10.0 py_0
toposort 1.5 py_3 conda-forge tornado 6.0.4 py37he774522_1
tqdm 4.47.0 py_0
urllib3 1.25.9 py_0
vc 14.1 h0510ff6_4
vs2015_runtime 14.16.27012 hf0eaf9b_3
werkzeug 0.16.1 py_0
wheel 0.34.2 py37_0
win_inet_pton 1.1.0 py37_0
wincertstore 0.2 py37_0
wrapt 1.12.1 py37he774522_1
xz 5.2.5 h62dcd97_0
yaml 0.2.5 he774522_0
zlib 1.2.11 h62dcd97_4
zstd 1.4.5 h04227a9_0 =============== State File ================= { "name": "lightweight", "sessions": { "1": { "timestamp": 1596959092.584746, "no_logs": false, "pingpong": false, "loss_names": { "a": [ "face_loss" ], "b": [ "face_loss" ] }, "batchsize": 12, "iterations": 201, "config": { "learning_rate": 5e-05 } } }, "lowest_avg_loss": { "a": 0.15506478816270827, "b": 0.12676110863685608 }, "iterations": 201, "inputs": { "face_in:0": [ 64, 64, 3 ] }, "training_size": 256, "config": { "coverage": 87.5, "mask_type": null, "mask_blur_kernel": 3, "mask_threshold": 4, "learn_mask": false, "icnr_init": false, "conv_aware_init": false, "reflect_padding": false, "penalized_mask_loss": true, "loss_function": "ssim", "learning_rate": 5e-05 } } ================= Configs ================== --------- .faceswap --------- backend: nvidia --------- convert.ini --------- [color.color_transfer] clip: True preserve_paper: True [color.manual_balance] colorspace: HSV balance_1: 0.0 balance_2: 0.0 balance_3: 0.0 contrast: 0.0 brightness: 0.0 [color.match_hist] threshold: 99.0 [mask.box_blend] type: gaussian distance: 11.0 radius: 5.0 passes: 1 [mask.mask_blend] type: normalized kernel_size: 3 passes: 4 threshold: 4 erosion: 0.0 [scaling.sharpen] method: unsharp_mask amount: 150 radius: 0.3 threshold: 5.0 [writer.ffmpeg] container: mp4 codec: libx264 crf: 23 preset: medium tune: none profile: auto level: auto [writer.gif] fps: 25 loop: 0 palettesize: 256 subrectangles: False [writer.opencv] format: png draw_transparent: False jpg_quality: 75 png_compress_level: 3 [writer.pillow] format: png draw_transparent: False optimize: False gif_interlace: True jpg_quality: 75 png_compress_level: 3 tif_compression: tiff_deflate --------- extract.ini --------- [global] allow_growth: False [align.fan] batch-size: 12 [detect.cv2_dnn] confidence: 50 [detect.mtcnn] minsize: 20 threshold_1: 0.6 threshold_2: 0.7 threshold_3: 0.7 scalefactor: 0.709 batch-size: 8 [detect.s3fd] confidence: 70 batch-size: 4 [mask.unet_dfl] batch-size: 8 [mask.vgg_clear] batch-size: 6 [mask.vgg_obstructed] batch-size: 2 --------- gui.ini --------- [global] fullscreen: False tab: extract options_panel_width: 30 console_panel_height: 20 icon_size: 14 font: default font_size: 9 autosave_last_session: prompt timeout: 120 auto_load_model_stats: True --------- train.ini --------- [global] coverage: 87.5 mask_type: none mask_blur_kernel: 3 mask_threshold: 4 learn_mask: False icnr_init: False conv_aware_init: False reflect_padding: False penalized_mask_loss: True loss_function: ssim learning_rate: 5e-05 [model.dfl_h128] lowmem: True [model.dfl_sae] input_size: 128 clipnorm: True architecture: df autoencoder_dims: 0 encoder_dims: 42 decoder_dims: 21 multiscale_decoder: False [model.dlight] features: best details: good output_size: 256 [model.original] lowmem: False [model.realface] input_size: 64 output_size: 128 dense_nodes: 1536 complexity_encoder: 128 complexity_decoder: 512 [model.unbalanced] input_size: 128 lowmem: False clipnorm: True nodes: 1024 complexity_encoder: 128 complexity_decoder_a: 384 complexity_decoder_b: 512 [model.villain] lowmem: False [trainer.original] preview_images: 14 zoom_amount: 5 rotation_range: 10 shift_range: 5 flip_chance: 50 color_lightness: 30 color_ab: 8 color_clahe_chance: 50 color_clahe_max_size: 4

Re: Crash after running for a few minutes

Posted: Sun Aug 09, 2020 9:43 am
by torzdf

Have you tried fixing your filenames as per the error message?

Error in thread (_run_1): Error while reading image. This is most likely caused by special characters in the filename: 'Z:\Desktop\AIhuanlian\xhssf\xhssf_000514_1 - 副本.png'