it crashes as soon as i try to train it and comes up with
Code: Select all
04/03/2020 23:10:23 CRITICAL An unexpected crash has occurred. Crash report written to 'C:\Users\helpo\faceswap\crash_report.2020.04.03.231023431555.log'. You MUST provide this file if seeking assistance. Please verify you are running the latest version of faceswap before reporting and the log says 04/03/2020 23:10:11 MainProcess _training_0 _config load_module DEBUG Adding defaults: (filename: unbalanced_defaults.py, module_path: plugins.train.model, plugin_type: model
04/03/2020 23:10:11 MainProcess _training_0 _config load_module DEBUG Importing defaults module: plugins.train.model.unbalanced_defaults
04/03/2020 23:10:11 MainProcess _training_0 config add_section DEBUG Add section: (title: 'model.unbalanced', info: 'An unbalanced model with adjustable input size options.\nThis is an unbalanced model so b>a swaps may not work well\n\nNB: Unless specifically stated, values changed here will only take effect when creating a new model.')
04/03/2020 23:10:11 MainProcess _training_0 config add_item DEBUG Add item: (section: 'model.unbalanced', title: 'input_size', datatype: '<class 'int'>', default: '128', info: 'Resolution (in pixels) of the image to train on.\nBE AWARE Larger resolution will dramatically increaseVRAM requirements.\nMake sure your resolution is divisible by 64 (e.g. 64, 128, 256 etc.).\nNB: Your faceset must be at least 1.6x larger than your required input size.\n(e.g. 160 is the maximum input size for a 256x256 faceset).', rounding: '64', min_max: (64, 512), choices: [], gui_radio: False, fixed: True, group: size)
04/03/2020 23:10:11 MainProcess _training_0 config add_item DEBUG Add item: (section: 'model.unbalanced', title: 'lowmem', datatype: '<class 'bool'>', default: 'False', info: 'Lower memory mode. Set to 'True' if having issues with VRAM useage.\nNB: Models with a changed lowmem mode are not compatible with each other.\nNB: lowmem will override cutom nodes and complexity settings.', rounding: 'None', min_max: None, choices: [], gui_radio: False, fixed: True, group: settings)
04/03/2020 23:10:11 MainProcess _training_0 config add_item DEBUG Add item: (section: 'model.unbalanced', title: 'clipnorm', datatype: '<class 'bool'>', default: 'True', info: 'Controls gradient clipping of the optimizer. Can prevent model corruption at the expense of VRAM.', rounding: 'None', min_max: None, choices: [], gui_radio: False, fixed: True, group: settings)
04/03/2020 23:10:11 MainProcess _training_0 config add_item DEBUG Add item: (section: 'model.unbalanced', title: 'nodes', datatype: '<class 'int'>', default: '1024', info: 'Number of nodes for decoder. Don't change this unless you know what you are doing!', rounding: '64', min_max: (512, 4096), choices: [], gui_radio: False, fixed: True, group: network)
04/03/2020 23:10:11 MainProcess _training_0 config add_item DEBUG Add item: (section: 'model.unbalanced', title: 'complexity_encoder', datatype: '<class 'int'>', default: '128', info: 'Encoder Convolution Layer Complexity. sensible ranges: 128 to 160.', rounding: '16', min_max: (64, 1024), choices: [], gui_radio: False, fixed: True, group: network)
04/03/2020 23:10:11 MainProcess _training_0 config add_item DEBUG Add item: (section: 'model.unbalanced', title: 'complexity_decoder_a', datatype: '<class 'int'>', default: '384', info: 'Decoder A Complexity.', rounding: '16', min_max: (64, 1024), choices: [], gui_radio: False, fixed: True, group: network)
04/03/2020 23:10:11 MainProcess _training_0 config add_item DEBUG Add item: (section: 'model.unbalanced', title: 'complexity_decoder_b', datatype: '<class 'int'>', default: '512', info: 'Decoder B Complexity.', rounding: '16', min_max: (64, 1024), choices: [], gui_radio: False, fixed: True, group: network)
04/03/2020 23:10:11 MainProcess _training_0 _config load_module DEBUG Added defaults: model.unbalanced
04/03/2020 23:10:11 MainProcess _training_0 _config load_module DEBUG Adding defaults: (filename: villain_defaults.py, module_path: plugins.train.model, plugin_type: model
04/03/2020 23:10:11 MainProcess _training_0 _config load_module DEBUG Importing defaults module: plugins.train.model.villain_defaults
04/03/2020 23:10:11 MainProcess _training_0 config add_section DEBUG Add section: (title: 'model.villain', info: 'A Higher resolution version of the Original Model by VillainGuy.\nExtremely VRAM heavy. Full model requires 9GB+ for batchsize 16\n\nNB: Unless specifically stated, values changed here will only take effect when creating a new model.')
04/03/2020 23:10:11 MainProcess _training_0 config add_item DEBUG Add item: (section: 'model.villain', title: 'lowmem', datatype: '<class 'bool'>', default: 'False', info: 'Lower memory mode. Set to 'True' if having issues with VRAM useage.\nNB: Models with a changed lowmem mode are not compatible with each other.', rounding: 'None', min_max: None, choices: [], gui_radio: False, fixed: True, group: settings)
04/03/2020 23:10:11 MainProcess _training_0 _config load_module DEBUG Added defaults: model.villain
04/03/2020 23:10:11 MainProcess _training_0 _config load_module DEBUG Adding defaults: (filename: original_defaults.py, module_path: plugins.train.trainer, plugin_type: trainer
04/03/2020 23:10:11 MainProcess _training_0 _config load_module DEBUG Importing defaults module: plugins.train.trainer.original_defaults
04/03/2020 23:10:11 MainProcess _training_0 config add_section DEBUG Add section: (title: 'trainer.original', info: 'Original Trainer Options.\nWARNING: The defaults for augmentation will be fine for 99.9% of use cases. Only change them if you absolutely know what you are doing!')
04/03/2020 23:10:11 MainProcess _training_0 config add_item DEBUG Add item: (section: 'trainer.original', title: 'preview_images', datatype: '<class 'int'>', default: '14', info: 'Number of sample faces to display for each side in the preview when training.', rounding: '2', min_max: (2, 16), choices: None, gui_radio: False, fixed: True, group: evaluation)
04/03/2020 23:10:11 MainProcess _training_0 config add_item DEBUG Add item: (section: 'trainer.original', title: 'zoom_amount', datatype: '<class 'int'>', default: '5', info: 'Percentage amount to randomly zoom each training image in and out.', rounding: '1', min_max: (0, 25), choices: None, gui_radio: False, fixed: True, group: image augmentation)
04/03/2020 23:10:11 MainProcess _training_0 config add_item DEBUG Add item: (section: 'trainer.original', title: 'rotation_range', datatype: '<class 'int'>', default: '10', info: 'Percentage amount to randomly rotate each training image.', rounding: '1', min_max: (0, 25), choices: None, gui_radio: False, fixed: True, group: image augmentation)
04/03/2020 23:10:11 MainProcess _training_0 config add_item DEBUG Add item: (section: 'trainer.original', title: 'shift_range', datatype: '<class 'int'>', default: '5', info: 'Percentage amount to randomly shift each training image horizontally and vertically.', rounding: '1', min_max: (0, 25), choices: None, gui_radio: False, fixed: True, group: image augmentation)
04/03/2020 23:10:11 MainProcess _training_0 config add_item DEBUG Add item: (section: 'trainer.original', title: 'flip_chance', datatype: '<class 'int'>', default: '50', info: 'Percentage chance to randomly flip each training image horizontally.\nNB: This is ignored if the 'no-flip' option is enabled', rounding: '1', min_max: (0, 75), choices: None, gui_radio: False, fixed: True, group: image augmentation)
04/03/2020 23:10:11 MainProcess _training_0 config add_item DEBUG Add item: (section: 'trainer.original', title: 'color_lightness', datatype: '<class 'int'>', default: '30', info: 'Percentage amount to randomly alter the lightness of each training image.\nNB: This is ignored if the 'no-augment-color' option is enabled', rounding: '1', min_max: (0, 75), choices: None, gui_radio: False, fixed: True, group: color augmentation)
04/03/2020 23:10:11 MainProcess _training_0 config add_item DEBUG Add item: (section: 'trainer.original', title: 'color_ab', datatype: '<class 'int'>', default: '8', info: 'Percentage amount to randomly alter the 'a' and 'b' colors of the L*a*b* color space of each training image.\nNB: This is ignored if the 'no-augment-color' option is enabled', rounding: '1', min_max: (0, 50), choices: None, gui_radio: False, fixed: True, group: color augmentation)
04/03/2020 23:10:11 MainProcess _training_0 config add_item DEBUG Add item: (section: 'trainer.original', title: 'color_clahe_chance', datatype: '<class 'int'>', default: '50', info: 'Percentage chance to perform Contrast Limited Adaptive Histogram Equalization on each training image.\nNB: This is ignored if the 'no-augment-color' option is enabled', rounding: '1', min_max: (0, 75), choices: None, gui_radio: False, fixed: False, group: color augmentation)
04/03/2020 23:10:11 MainProcess _training_0 config add_item DEBUG Add item: (section: 'trainer.original', title: 'color_clahe_max_size', datatype: '<class 'int'>', default: '4', info: 'The grid size dictates how much Contrast Limited Adaptive Histogram Equalization is performed on any training image selected for clahe. Contrast will be applied randomly with a gridsize of 0 up to the maximum. This value is a multiplier calculated from the training image size.\nNB: This is ignored if the 'no-augment-color' option is enabled', rounding: '1', min_max: (1, 8), choices: None, gui_radio: False, fixed: True, group: color augmentation)
04/03/2020 23:10:11 MainProcess _training_0 _config load_module DEBUG Added defaults: trainer.original
04/03/2020 23:10:11 MainProcess _training_0 config handle_config DEBUG Handling config
04/03/2020 23:10:11 MainProcess _training_0 config check_exists DEBUG Config file exists: 'C:\Users\helpo\faceswap\config\train.ini'
04/03/2020 23:10:11 MainProcess _training_0 config load_config VERBOSE Loading config: 'C:\Users\helpo\faceswap\config\train.ini'
04/03/2020 23:10:11 MainProcess _training_0 config validate_config DEBUG Validating config
04/03/2020 23:10:11 MainProcess _training_0 config check_config_change DEBUG Default config has not changed
04/03/2020 23:10:11 MainProcess _training_0 config check_config_choices DEBUG Checking config choices
04/03/2020 23:10:11 MainProcess _training_0 config check_config_choices DEBUG Checked config choices
04/03/2020 23:10:11 MainProcess _training_0 config validate_config DEBUG Validated config
04/03/2020 23:10:11 MainProcess _training_0 config handle_config DEBUG Handled config
04/03/2020 23:10:11 MainProcess _training_0 config __init__ DEBUG Initialized: Config
04/03/2020 23:10:11 MainProcess _training_0 config get DEBUG Getting config item: (section: 'global', option: 'learning_rate')
04/03/2020 23:10:11 MainProcess _training_0 config get DEBUG Returning item: (type: <class 'float'>, value: 5e-05)
04/03/2020 23:10:11 MainProcess _training_0 config changeable_items DEBUG Alterable for existing models: {'learning_rate': 5e-05}
04/03/2020 23:10:11 MainProcess _training_0 _base __init__ DEBUG Initializing State: (model_dir: 'D:\finnish\original_logs', model_name: 'original', config_changeable_items: '{'learning_rate': 5e-05}', no_logs: False, pingpong: False, training_image_size: '270'
04/03/2020 23:10:11 MainProcess _training_0 serializer get_serializer DEBUG <lib.serializer._JSONSerializer object at 0x000001E2159613C8>
04/03/2020 23:10:11 MainProcess _training_0 _base load DEBUG Loading State
04/03/2020 23:10:11 MainProcess _training_0 _base load INFO No existing state file found. Generating.
04/03/2020 23:10:11 MainProcess _training_0 _base new_session_id DEBUG 1
04/03/2020 23:10:11 MainProcess _training_0 _base create_new_session DEBUG Creating new session. id: 1
04/03/2020 23:10:11 MainProcess _training_0 _base __init__ DEBUG Initialized State:
04/03/2020 23:10:11 MainProcess _training_0 nn_blocks __init__ DEBUG Initializing NNBlocks: (use_subpixel: False, use_icnr_init: False, use_convaware_init: False, use_reflect_padding: False, first_run: True)
04/03/2020 23:10:11 MainProcess _training_0 nn_blocks __init__ DEBUG Initialized NNBlocks
04/03/2020 23:10:11 MainProcess _training_0 _base name DEBUG model name: 'original'
04/03/2020 23:10:11 MainProcess _training_0 _base rename_legacy DEBUG Renaming legacy files
04/03/2020 23:10:11 MainProcess _training_0 _base name DEBUG model name: 'original'
04/03/2020 23:10:11 MainProcess _training_0 _base rename_legacy DEBUG No legacy files to rename
04/03/2020 23:10:11 MainProcess _training_0 _base load_state_info DEBUG Loading Input Shape from State file
04/03/2020 23:10:11 MainProcess _training_0 _base load_state_info DEBUG No input shapes saved. Using model config
04/03/2020 23:10:11 MainProcess _training_0 _base calculate_coverage_ratio DEBUG Requested coverage_ratio: 0.6875
04/03/2020 23:10:11 MainProcess _training_0 _base calculate_coverage_ratio DEBUG Final coverage_ratio: 0.6814814814814815
04/03/2020 23:10:11 MainProcess _training_0 _base __init__ DEBUG training_opts: {'alignments': {'a': 'C:\\Users\\helpo\\OneDrive\\Pictures\\photo\\alignments.fsa', 'b': 'C:\\Users\\helpo\\OneDrive\\Pictures\\photo b\\alignments.fsa'}, 'preview_scaling': 0.5, 'warp_to_landmarks': False, 'augment_color': True, 'no_flip': False, 'pingpong': False, 'snapshot_interval': 25000, 'training_size': 270, 'no_logs': False, 'coverage_ratio': 0.6814814814814815, 'mask_type': None, 'mask_blur_kernel': 3, 'mask_threshold': 4, 'learn_mask': False, 'penalized_mask_loss': False}
04/03/2020 23:10:11 MainProcess _training_0 _base multiple_models_in_folder DEBUG model_files: [], retval: False
04/03/2020 23:10:11 MainProcess _training_0 original add_networks DEBUG Adding networks
04/03/2020 23:10:11 MainProcess _training_0 module_wrapper _tfmw_add_deprecation_warning DEBUG From C:\Users\helpo\MiniConda3\envs\faceswap\lib\site-packages\keras\backend\tensorflow_backend.py:74: The name tf.get_default_graph is deprecated. Please use tf.compat.v1.get_default_graph instead.\n
04/03/2020 23:10:11 MainProcess _training_0 module_wrapper _tfmw_add_deprecation_warning DEBUG From C:\Users\helpo\MiniConda3\envs\faceswap\lib\site-packages\keras\backend\tensorflow_backend.py:517: The name tf.placeholder is deprecated. Please use tf.compat.v1.placeholder instead.\n
04/03/2020 23:10:11 MainProcess _training_0 nn_blocks upscale DEBUG inp: Tensor("input_1:0", shape=(?, 8, 8, 512), dtype=float32), filters: 256, kernel_size: 3, use_instance_norm: False, kwargs: {})
04/03/2020 23:10:11 MainProcess _training_0 nn_blocks get_name DEBUG Generating block name: upscale_8_0
04/03/2020 23:10:11 MainProcess _training_0 nn_blocks set_default_initializer DEBUG Set default kernel_initializer to: <keras.initializers.VarianceScaling object at 0x000001E2152DF548>
04/03/2020 23:10:11 MainProcess _training_0 nn_blocks conv2d DEBUG inp: Tensor("input_1:0", shape=(?, 8, 8, 512), dtype=float32), filters: 1024, kernel_size: 3, strides: (1, 1), padding: same, kwargs: {'name': 'upscale_8_0_conv2d', 'kernel_initializer': <keras.initializers.VarianceScaling object at 0x000001E2152DF548>})
04/03/2020 23:10:11 MainProcess _training_0 nn_blocks set_default_initializer DEBUG Using model specified initializer: <keras.initializers.VarianceScaling object at 0x000001E2152DF548>
04/03/2020 23:10:11 MainProcess _training_0 module_wrapper _tfmw_add_deprecation_warning DEBUG From C:\Users\helpo\MiniConda3\envs\faceswap\lib\site-packages\keras\backend\tensorflow_backend.py:4138: The name tf.random_uniform is deprecated. Please use tf.random.uniform instead.\n
04/03/2020 23:10:11 MainProcess _training_0 nn_blocks upscale DEBUG inp: Tensor("upscale_8_0_pixelshuffler/Reshape_1:0", shape=(?, 16, 16, 256), dtype=float32), filters: 128, kernel_size: 3, use_instance_norm: False, kwargs: {})
04/03/2020 23:10:11 MainProcess _training_0 nn_blocks get_name DEBUG Generating block name: upscale_16_0
04/03/2020 23:10:11 MainProcess _training_0 nn_blocks set_default_initializer DEBUG Set default kernel_initializer to: <keras.initializers.VarianceScaling object at 0x000001E215992608>
04/03/2020 23:10:11 MainProcess _training_0 nn_blocks conv2d DEBUG inp: Tensor("upscale_8_0_pixelshuffler/Reshape_1:0", shape=(?, 16, 16, 256), dtype=float32), filters: 512, kernel_size: 3, strides: (1, 1), padding: same, kwargs: {'name': 'upscale_16_0_conv2d', 'kernel_initializer': <keras.initializers.VarianceScaling object at 0x000001E215992608>})
04/03/2020 23:10:11 MainProcess _training_0 nn_blocks set_default_initializer DEBUG Using model specified initializer: <keras.initializers.VarianceScaling object at 0x000001E215992608>
04/03/2020 23:10:11 MainProcess _training_0 nn_blocks upscale DEBUG inp: Tensor("upscale_16_0_pixelshuffler/Reshape_1:0", shape=(?, 32, 32, 128), dtype=float32), filters: 64, kernel_size: 3, use_instance_norm: False, kwargs: {})
04/03/2020 23:10:11 MainProcess _training_0 nn_blocks get_name DEBUG Generating block name: upscale_32_0
04/03/2020 23:10:11 MainProcess _training_0 nn_blocks set_default_initializer DEBUG Set default kernel_initializer to: <keras.initializers.VarianceScaling object at 0x000001E2159AB388>
04/03/2020 23:10:11 MainProcess _training_0 nn_blocks conv2d DEBUG inp: Tensor("upscale_16_0_pixelshuffler/Reshape_1:0", shape=(?, 32, 32, 128), dtype=float32), filters: 256, kernel_size: 3, strides: (1, 1), padding: same, kwargs: {'name': 'upscale_32_0_conv2d', 'kernel_initializer': <keras.initializers.VarianceScaling object at 0x000001E2159AB388>})
04/03/2020 23:10:11 MainProcess _training_0 nn_blocks set_default_initializer DEBUG Using model specified initializer: <keras.initializers.VarianceScaling object at 0x000001E2159AB388>
04/03/2020 23:10:11 MainProcess _training_0 nn_blocks conv2d DEBUG inp: Tensor("upscale_32_0_pixelshuffler/Reshape_1:0", shape=(?, 64, 64, 64), dtype=float32), filters: 3, kernel_size: 5, strides: (1, 1), padding: same, kwargs: {'activation': 'sigmoid', 'name': 'face_out'})
04/03/2020 23:10:11 MainProcess _training_0 nn_blocks set_default_initializer DEBUG Set default kernel_initializer to: <keras.initializers.VarianceScaling object at 0x000001E2159AB748>
04/03/2020 23:10:11 MainProcess _training_0 _base add_network DEBUG network_type: 'decoder', side: 'a', network: '<keras.engine.training.Model object at 0x000001E2159C07C8>', is_output: True
04/03/2020 23:10:11 MainProcess _training_0 _base name DEBUG model name: 'original'
04/03/2020 23:10:11 MainProcess _training_0 _base add_network DEBUG name: 'decoder_a', filename: 'original_decoder_A.h5'
04/03/2020 23:10:11 MainProcess _training_0 _base __init__ DEBUG Initializing NNMeta: (filename: 'D:\finnish\original_logs\original_decoder_A.h5', network_type: 'decoder', side: 'a', network: <keras.engine.training.Model object at 0x000001E2159C07C8>, is_output: True
04/03/2020 23:10:11 MainProcess _training_0 module_wrapper _tfmw_add_deprecation_warning DEBUG From C:\Users\helpo\MiniConda3\envs\faceswap\lib\site-packages\keras\backend\tensorflow_backend.py:174: The name tf.get_default_session is deprecated. Please use tf.compat.v1.get_default_session instead.\n
04/03/2020 23:10:11 MainProcess _training_0 module_wrapper _tfmw_add_deprecation_warning DEBUG From C:\Users\helpo\MiniConda3\envs\faceswap\lib\site-packages\keras\backend\tensorflow_backend.py:181: The name tf.ConfigProto is deprecated. Please use tf.compat.v1.ConfigProto instead.\n
04/03/2020 23:10:11 MainProcess _training_0 module_wrapper _tfmw_add_deprecation_warning DEBUG From C:\Users\helpo\MiniConda3\envs\faceswap\lib\site-packages\keras\backend\tensorflow_backend.py:186: The name tf.Session is deprecated. Please use tf.compat.v1.Session instead.\n
04/03/2020 23:10:12 MainProcess _training_0 module_wrapper _tfmw_add_deprecation_warning DEBUG From C:\Users\helpo\MiniConda3\envs\faceswap\lib\site-packages\keras\backend\tensorflow_backend.py:190: The name tf.global_variables is deprecated. Please use tf.compat.v1.global_variables instead.\n
04/03/2020 23:10:12 MainProcess _training_0 module_wrapper _tfmw_add_deprecation_warning DEBUG From C:\Users\helpo\MiniConda3\envs\faceswap\lib\site-packages\keras\backend\tensorflow_backend.py:199: The name tf.is_variable_initialized is deprecated. Please use tf.compat.v1.is_variable_initialized instead.\n
04/03/2020 23:10:12 MainProcess _training_0 module_wrapper _tfmw_add_deprecation_warning DEBUG From C:\Users\helpo\MiniConda3\envs\faceswap\lib\site-packages\keras\backend\tensorflow_backend.py:206: The name tf.variables_initializer is deprecated. Please use tf.compat.v1.variables_initializer instead.\n
04/03/2020 23:10:22 MainProcess _training_0 multithreading run DEBUG Error in thread (_training_0): Dst tensor is not initialized.\n [[node upscale_16_0_conv2d/kernel (defined at \MiniConda3\envs\faceswap\lib\site-packages\tensorflow_core\python\framework\ops.py:1748) ]]\n\nOriginal stack trace for 'upscale_16_0_conv2d/kernel':\n File "\MiniConda3\envs\faceswap\lib\threading.py", line 890, in _bootstrap\n self._bootstrap_inner()\n File "\MiniConda3\envs\faceswap\lib\threading.py", line 926, in _bootstrap_inner\n self.run()\n File "\faceswap\lib\multithreading.py", line 37, in run\n self._target(*self._args, **self._kwargs)\n File "\faceswap\scripts\train.py", line 212, in _training\n model = self._load_model()\n File "\faceswap\scripts\train.py", line 253, in _load_model\n predict=False)\n File "\faceswap\plugins\train\model\original.py", line 25, in __init__\n super().__init__(*args, **kwargs)\n File "\faceswap\plugins\train\model\_base.py", line 126, in __init__\n self.build()\n File "\faceswap\plugins\train\model\_base.py", line 244, in build\n self.add_networks()\n File "\faceswap\plugins\train\model\original.py", line 31, in add_networks\n self.add_network("decoder", "a", self.decoder(), is_output=True)\n File "\faceswap\plugins\train\model\original.py", line 67, in decoder\n var_x = self.blocks.upscale(var_x, 128)\n File "\faceswap\lib\model\nn_blocks.py", line 137, in upscale\n **kwargs)\n File "\faceswap\lib\model\nn_blocks.py", line 90, in conv2d\n **kwargs)(inp)\n File "\MiniConda3\envs\faceswap\lib\site-packages\keras\engine\base_layer.py", line 431, in __call__\n self.build(unpack_singleton(input_shapes))\n File "\MiniConda3\envs\faceswap\lib\site-packages\keras\layers\convolutional.py", line 141, in build\n constraint=self.kernel_constraint)\n File "\MiniConda3\envs\faceswap\lib\site-packages\keras\legacy\interfaces.py", line 91, in wrapper\n return func(*args, **kwargs)\n File "\MiniConda3\envs\faceswap\lib\site-packages\keras\engine\base_layer.py", line 252, in add_weight\n constraint=constraint)\n File "\MiniConda3\envs\faceswap\lib\site-packages\keras\backend\tensorflow_backend.py", line 402, in variable\n v = tf.Variable(value, dtype=tf.as_dtype(dtype), name=name)\n File "\MiniConda3\envs\faceswap\lib\site-packages\tensorflow_core\python\ops\variables.py", line 258, in __call__\n return cls._variable_v1_call(*args, **kwargs)\n File "\MiniConda3\envs\faceswap\lib\site-packages\tensorflow_core\python\ops\variables.py", line 219, in _variable_v1_call\n shape=shape)\n File "\MiniConda3\envs\faceswap\lib\site-packages\tensorflow_core\python\ops\variables.py", line 197, in <lambda>\n previous_getter = lambda **kwargs: default_variable_creator(None, **kwargs)\n File "\MiniConda3\envs\faceswap\lib\site-packages\tensorflow_core\python\ops\variable_scope.py", line 2519, in default_variable_creator\n shape=shape)\n File "\MiniConda3\envs\faceswap\lib\site-packages\tensorflow_core\python\ops\variables.py", line 262, in __call__\n return super(VariableMetaclass, cls).__call__(*args, **kwargs)\n File "\MiniConda3\envs\faceswap\lib\site-packages\tensorflow_core\python\ops\variables.py", line 1688, in __init__\n shape=shape)\n File "\MiniConda3\envs\faceswap\lib\site-packages\tensorflow_core\python\ops\variables.py", line 1846, in _init_from_args\n shape, self._initial_value.dtype.base_dtype, name=name)\n File "\MiniConda3\envs\faceswap\lib\site-packages\tensorflow_core\python\ops\state_ops.py", line 79, in variable_op_v2\n shared_name=shared_name)\n File "\MiniConda3\envs\faceswap\lib\site-packages\tensorflow_core\python\ops\gen_state_ops.py", line 1621, in variable_v2\n shared_name=shared_name, name=name)\n File "\MiniConda3\envs\faceswap\lib\site-packages\tensorflow_core\python\framework\op_def_library.py", line 794, in _apply_op_helper\n op_def=op_def)\n File "\MiniConda3\envs\faceswap\lib\site-packages\tensorflow_core\python\util\deprecation.py", line 507, in new_func\n return func(*args, **kwargs)\n File "\MiniConda3\envs\faceswap\lib\site-packages\tensorflow_core\python\framework\ops.py", line 3357, in create_op\n attrs, op_def, compute_device)\n File "\MiniConda3\envs\faceswap\lib\site-packages\tensorflow_core\python\framework\ops.py", line 3426, in _create_op_internal\n op_def=op_def)\n File "\MiniConda3\envs\faceswap\lib\site-packages\tensorflow_core\python\framework\ops.py", line 1748, in __init__\n self._traceback = tf_stack.extract_stack()\n
04/03/2020 23:10:23 MainProcess MainThread train _monitor DEBUG Thread error detected
04/03/2020 23:10:23 MainProcess MainThread train _monitor DEBUG Closed Monitor
04/03/2020 23:10:23 MainProcess MainThread train _end_thread DEBUG Ending Training thread
04/03/2020 23:10:23 MainProcess MainThread train _end_thread CRITICAL Error caught! Exiting...
04/03/2020 23:10:23 MainProcess MainThread multithreading join DEBUG Joining Threads: '_training'
04/03/2020 23:10:23 MainProcess MainThread multithreading join DEBUG Joining Thread: '_training_0'
04/03/2020 23:10:23 MainProcess MainThread multithreading join ERROR Caught exception in thread: '_training_0'
Traceback (most recent call last):
File "C:\Users\helpo\MiniConda3\envs\faceswap\lib\site-packages\tensorflow_core\python\client\session.py", line 1365, in _do_call
return fn(*args)
File "C:\Users\helpo\MiniConda3\envs\faceswap\lib\site-packages\tensorflow_core\python\client\session.py", line 1350, in _run_fn
target_list, run_metadata)
File "C:\Users\helpo\MiniConda3\envs\faceswap\lib\site-packages\tensorflow_core\python\client\session.py", line 1443, in _call_tf_sessionrun
run_metadata)
tensorflow.python.framework.errors_impl.InternalError: Dst tensor is not initialized.
[[{{node upscale_16_0_conv2d/kernel}}]]
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "C:\Users\helpo\faceswap\lib\cli.py", line 128, in execute_script
process.process()
File "C:\Users\helpo\faceswap\scripts\train.py", line 159, in process
self._end_thread(thread, err)
File "C:\Users\helpo\faceswap\scripts\train.py", line 199, in _end_thread
thread.join()
File "C:\Users\helpo\faceswap\lib\multithreading.py", line 121, in join
raise thread.err[1].with_traceback(thread.err[2])
File "C:\Users\helpo\faceswap\lib\multithreading.py", line 37, in run
self._target(*self._args, **self._kwargs)
File "C:\Users\helpo\faceswap\scripts\train.py", line 224, in _training
raise err
File "C:\Users\helpo\faceswap\scripts\train.py", line 212, in _training
model = self._load_model()
File "C:\Users\helpo\faceswap\scripts\train.py", line 253, in _load_model
predict=False)
File "C:\Users\helpo\faceswap\plugins\train\model\original.py", line 25, in __init__
super().__init__(*args, **kwargs)
File "C:\Users\helpo\faceswap\plugins\train\model\_base.py", line 126, in __init__
self.build()
File "C:\Users\helpo\faceswap\plugins\train\model\_base.py", line 244, in build
self.add_networks()
File "C:\Users\helpo\faceswap\plugins\train\model\original.py", line 31, in add_networks
self.add_network("decoder", "a", self.decoder(), is_output=True)
File "C:\Users\helpo\faceswap\plugins\train\model\_base.py", line 319, in add_network
is_output)
File "C:\Users\helpo\faceswap\plugins\train\model\_base.py", line 796, in __init__
self.weights = network.get_weights() # For pingpong restore
File "C:\Users\helpo\MiniConda3\envs\faceswap\lib\site-packages\keras\engine\network.py", line 492, in get_weights
return K.batch_get_value(weights)
File "C:\Users\helpo\MiniConda3\envs\faceswap\lib\site-packages\keras\backend\tensorflow_backend.py", line 2420, in batch_get_value
return get_session().run(ops)
File "C:\Users\helpo\MiniConda3\envs\faceswap\lib\site-packages\tensorflow_core\python\client\session.py", line 956, in run
run_metadata_ptr)
File "C:\Users\helpo\MiniConda3\envs\faceswap\lib\site-packages\tensorflow_core\python\client\session.py", line 1180, in _run
feed_dict_tensor, options, run_metadata)
File "C:\Users\helpo\MiniConda3\envs\faceswap\lib\site-packages\tensorflow_core\python\client\session.py", line 1359, in _do_run
run_metadata)
File "C:\Users\helpo\MiniConda3\envs\faceswap\lib\site-packages\tensorflow_core\python\client\session.py", line 1384, in _do_call
raise type(e)(node_def, op, message)
tensorflow.python.framework.errors_impl.InternalError: Dst tensor is not initialized.
[[node upscale_16_0_conv2d/kernel (defined at \MiniConda3\envs\faceswap\lib\site-packages\tensorflow_core\python\framework\ops.py:1748) ]]
Original stack trace for 'upscale_16_0_conv2d/kernel':
File "\MiniConda3\envs\faceswap\lib\threading.py", line 890, in _bootstrap
self._bootstrap_inner()
File "\MiniConda3\envs\faceswap\lib\threading.py", line 926, in _bootstrap_inner
self.run()
File "\faceswap\lib\multithreading.py", line 37, in run
self._target(*self._args, **self._kwargs)
File "\faceswap\scripts\train.py", line 212, in _training
model = self._load_model()
File "\faceswap\scripts\train.py", line 253, in _load_model
predict=False)
File "\faceswap\plugins\train\model\original.py", line 25, in __init__
super().__init__(*args, **kwargs)
File "\faceswap\plugins\train\model\_base.py", line 126, in __init__
self.build()
File "\faceswap\plugins\train\model\_base.py", line 244, in build
self.add_networks()
File "\faceswap\plugins\train\model\original.py", line 31, in add_networks
self.add_network("decoder", "a", self.decoder(), is_output=True)
File "\faceswap\plugins\train\model\original.py", line 67, in decoder
var_x = self.blocks.upscale(var_x, 128)
File "\faceswap\lib\model\nn_blocks.py", line 137, in upscale
**kwargs)
File "\faceswap\lib\model\nn_blocks.py", line 90, in conv2d
**kwargs)(inp)
File "\MiniConda3\envs\faceswap\lib\site-packages\keras\engine\base_layer.py", line 431, in __call__
self.build(unpack_singleton(input_shapes))
File "\MiniConda3\envs\faceswap\lib\site-packages\keras\layers\convolutional.py", line 141, in build
constraint=self.kernel_constraint)
File "\MiniConda3\envs\faceswap\lib\site-packages\keras\legacy\interfaces.py", line 91, in wrapper
return func(*args, **kwargs)
File "\MiniConda3\envs\faceswap\lib\site-packages\keras\engine\base_layer.py", line 252, in add_weight
constraint=constraint)
File "\MiniConda3\envs\faceswap\lib\site-packages\keras\backend\tensorflow_backend.py", line 402, in variable
v = tf.Variable(value, dtype=tf.as_dtype(dtype), name=name)
File "\MiniConda3\envs\faceswap\lib\site-packages\tensorflow_core\python\ops\variables.py", line 258, in __call__
return cls._variable_v1_call(*args, **kwargs)
File "\MiniConda3\envs\faceswap\lib\site-packages\tensorflow_core\python\ops\variables.py", line 219, in _variable_v1_call
shape=shape)
File "\MiniConda3\envs\faceswap\lib\site-packages\tensorflow_core\python\ops\variables.py", line 197, in <lambda>
previous_getter = lambda **kwargs: default_variable_creator(None, **kwargs)
File "\MiniConda3\envs\faceswap\lib\site-packages\tensorflow_core\python\ops\variable_scope.py", line 2519, in default_variable_creator
shape=shape)
File "\MiniConda3\envs\faceswap\lib\site-packages\tensorflow_core\python\ops\variables.py", line 262, in __call__
return super(VariableMetaclass, cls).__call__(*args, **kwargs)
File "\MiniConda3\envs\faceswap\lib\site-packages\tensorflow_core\python\ops\variables.py", line 1688, in __init__
shape=shape)
File "\MiniConda3\envs\faceswap\lib\site-packages\tensorflow_core\python\ops\variables.py", line 1846, in _init_from_args
shape, self._initial_value.dtype.base_dtype, name=name)
File "\MiniConda3\envs\faceswap\lib\site-packages\tensorflow_core\python\ops\state_ops.py", line 79, in variable_op_v2
shared_name=shared_name)
File "\MiniConda3\envs\faceswap\lib\site-packages\tensorflow_core\python\ops\gen_state_ops.py", line 1621, in variable_v2
shared_name=shared_name, name=name)
File "\MiniConda3\envs\faceswap\lib\site-packages\tensorflow_core\python\framework\op_def_library.py", line 794, in _apply_op_helper
op_def=op_def)
File "\MiniConda3\envs\faceswap\lib\site-packages\tensorflow_core\python\util\deprecation.py", line 507, in new_func
return func(*args, **kwargs)
File "\MiniConda3\envs\faceswap\lib\site-packages\tensorflow_core\python\framework\ops.py", line 3357, in create_op
attrs, op_def, compute_device)
File "\MiniConda3\envs\faceswap\lib\site-packages\tensorflow_core\python\framework\ops.py", line 3426, in _create_op_internal
op_def=op_def)
File "\MiniConda3\envs\faceswap\lib\site-packages\tensorflow_core\python\framework\ops.py", line 1748, in __init__
self._traceback = tf_stack.extract_stack()
Exception occured trying to retrieve sysinfo: [WinError 1455] The paging file is too small for this operation to complete
