Code: Select all
Loading...
Setting Faceswap backend to AMD
02/28/2021 07:53:43 INFO Log level set to: VERBOSE
02/28/2021 07:53:43 INFO Setting up for PlaidML
02/28/2021 07:53:43 VERBOSE Setting Keras Backend to PlaidML
02/28/2021 07:53:43 INFO Using GPU(s): ['Advanced Micro Devices, Inc. - Ellesmere (experimental)', 'Advanced Micro Devices, Inc. - Ellesmere (supported)']
02/28/2021 07:53:43 INFO Successfully set up for PlaidML
2021-02-28 07:53:43.158263: W tensorflow/stream_executor/platform/default/dso_loader.cc:55] Could not load dynamic library 'cudart64_101.dll'; dlerror: cudart64_101.dll not found
2021-02-28 07:53:43.158412: I tensorflow/stream_executor/cuda/cudart_stub.cc:29] Ignore above cudart dlerror if you do not have a GPU set up on your machine.
02/28/2021 07:53:44 INFO Model A Directory: D:\sf\x8\fc
02/28/2021 07:53:44 INFO Model B Directory: D:\sf\fc\fc_train
02/28/2021 07:53:44 INFO Training data directory: D:\sf\x8\model
02/28/2021 07:53:44 INFO ===================================================
02/28/2021 07:53:44 INFO Starting
02/28/2021 07:53:44 INFO Press 'Stop' to save and quit
02/28/2021 07:53:44 INFO ===================================================
02/28/2021 07:53:45 INFO Loading data, this may take a while...
02/28/2021 07:53:45 INFO Loading Model from Original plugin...
Using plaidml.keras.backend backend.
02/28/2021 07:53:45 VERBOSE Loading config: 'C:\Users\faceswap_amd\config\train.ini'
02/28/2021 07:53:45 VERBOSE Loading config: 'C:\Users\faceswap_amd\config\train.ini'
02/28/2021 07:53:45 INFO No existing state file found. Generating.
02/28/2021 07:53:45 INFO Opening device "opencl_amd_ellesmere.0"
02/28/2021 07:53:46 VERBOSE Using Adam optimizer
02/28/2021 07:53:46 VERBOSE __________________________________________________________________________________________________
02/28/2021 07:53:46 VERBOSE Layer (type) Output Shape Param # Connected to
02/28/2021 07:53:46 VERBOSE ==================================================================================================
02/28/2021 07:53:46 VERBOSE face_in_a (InputLayer) (None, 64, 64, 3) 0
02/28/2021 07:53:46 VERBOSE __________________________________________________________________________________________________
02/28/2021 07:53:46 VERBOSE face_in_b (InputLayer) (None, 64, 64, 3) 0
02/28/2021 07:53:46 VERBOSE __________________________________________________________________________________________________
02/28/2021 07:53:46 VERBOSE encoder (Model) (None, 8, 8, 512) 69662976 face_in_a[0][0]
02/28/2021 07:53:46 VERBOSE face_in_b[0][0]
02/28/2021 07:53:46 VERBOSE __________________________________________________________________________________________________
02/28/2021 07:53:46 VERBOSE decoder_a (Model) (None, 64, 64, 3) 6199747 encoder[1][0]
02/28/2021 07:53:46 VERBOSE __________________________________________________________________________________________________
02/28/2021 07:53:46 VERBOSE decoder_b (Model) (None, 64, 64, 3) 6199747 encoder[2][0]
02/28/2021 07:53:46 VERBOSE ==================================================================================================
02/28/2021 07:53:46 VERBOSE Total params: 82,062,470
02/28/2021 07:53:46 VERBOSE Trainable params: 82,062,470
02/28/2021 07:53:46 VERBOSE Non-trainable params: 0
02/28/2021 07:53:46 VERBOSE __________________________________________________________________________________________________
02/28/2021 07:53:46 VERBOSE _________________________________________________________________
02/28/2021 07:53:46 VERBOSE Layer (type) Output Shape Param #
02/28/2021 07:53:46 VERBOSE =================================================================
02/28/2021 07:53:46 VERBOSE input_1 (InputLayer) (None, 64, 64, 3) 0
02/28/2021 07:53:46 VERBOSE _________________________________________________________________
02/28/2021 07:53:46 VERBOSE conv_128_0_conv2d (Conv2D) (None, 32, 32, 128) 9728
02/28/2021 07:53:46 VERBOSE _________________________________________________________________
02/28/2021 07:53:46 VERBOSE conv_128_0_leakyrelu (LeakyR (None, 32, 32, 128) 0
02/28/2021 07:53:46 VERBOSE _________________________________________________________________
02/28/2021 07:53:46 VERBOSE conv_256_0_conv2d (Conv2D) (None, 16, 16, 256) 819456
02/28/2021 07:53:46 VERBOSE _________________________________________________________________
02/28/2021 07:53:46 VERBOSE conv_256_0_leakyrelu (LeakyR (None, 16, 16, 256) 0
02/28/2021 07:53:46 VERBOSE _________________________________________________________________
02/28/2021 07:53:46 VERBOSE conv_512_0_conv2d (Conv2D) (None, 8, 8, 512) 3277312
02/28/2021 07:53:46 VERBOSE _________________________________________________________________
02/28/2021 07:53:46 VERBOSE conv_512_0_leakyrelu (LeakyR (None, 8, 8, 512) 0
02/28/2021 07:53:46 VERBOSE _________________________________________________________________
02/28/2021 07:53:46 VERBOSE conv_1024_0_conv2d (Conv2D) (None, 4, 4, 1024) 13108224
02/28/2021 07:53:46 VERBOSE _________________________________________________________________
02/28/2021 07:53:46 VERBOSE conv_1024_0_leakyrelu (Leaky (None, 4, 4, 1024) 0
02/28/2021 07:53:46 VERBOSE _________________________________________________________________
02/28/2021 07:53:46 VERBOSE flatten_1 (Flatten) (None, 16384) 0
02/28/2021 07:53:46 VERBOSE _________________________________________________________________
02/28/2021 07:53:46 VERBOSE dense_1 (Dense) (None, 1024) 16778240
02/28/2021 07:53:46 VERBOSE _________________________________________________________________
02/28/2021 07:53:46 VERBOSE dense_2 (Dense) (None, 16384) 16793600
02/28/2021 07:53:46 VERBOSE _________________________________________________________________
02/28/2021 07:53:46 VERBOSE reshape_1 (Reshape) (None, 4, 4, 1024) 0
02/28/2021 07:53:46 VERBOSE _________________________________________________________________
02/28/2021 07:53:46 VERBOSE upscale_512_0_conv2d_conv2d (None, 4, 4, 2048) 18876416
02/28/2021 07:53:46 VERBOSE _________________________________________________________________
02/28/2021 07:53:46 VERBOSE upscale_512_0_conv2d_leakyre (None, 4, 4, 2048) 0
02/28/2021 07:53:46 VERBOSE _________________________________________________________________
02/28/2021 07:53:46 VERBOSE upscale_512_0_pixelshuffler (None, 8, 8, 512) 0
02/28/2021 07:53:46 VERBOSE =================================================================
02/28/2021 07:53:46 VERBOSE Total params: 69,662,976
02/28/2021 07:53:46 VERBOSE Trainable params: 69,662,976
02/28/2021 07:53:46 VERBOSE Non-trainable params: 0
02/28/2021 07:53:46 VERBOSE _________________________________________________________________
02/28/2021 07:53:46 VERBOSE _________________________________________________________________
02/28/2021 07:53:46 VERBOSE Layer (type) Output Shape Param #
02/28/2021 07:53:46 VERBOSE =================================================================
02/28/2021 07:53:46 VERBOSE input_2 (InputLayer) (None, 8, 8, 512) 0
02/28/2021 07:53:46 VERBOSE _________________________________________________________________
02/28/2021 07:53:46 VERBOSE upscale_256_0_conv2d_conv2d (None, 8, 8, 1024) 4719616
02/28/2021 07:53:46 VERBOSE _________________________________________________________________
02/28/2021 07:53:46 VERBOSE upscale_256_0_conv2d_leakyre (None, 8, 8, 1024) 0
02/28/2021 07:53:46 VERBOSE _________________________________________________________________
02/28/2021 07:53:46 VERBOSE upscale_256_0_pixelshuffler (None, 16, 16, 256) 0
02/28/2021 07:53:46 VERBOSE _________________________________________________________________
02/28/2021 07:53:46 VERBOSE upscale_128_0_conv2d_conv2d (None, 16, 16, 512) 1180160
02/28/2021 07:53:46 VERBOSE _________________________________________________________________
02/28/2021 07:53:46 VERBOSE upscale_128_0_conv2d_leakyre (None, 16, 16, 512) 0
02/28/2021 07:53:46 VERBOSE _________________________________________________________________
02/28/2021 07:53:46 VERBOSE upscale_128_0_pixelshuffler (None, 32, 32, 128) 0
02/28/2021 07:53:46 VERBOSE _________________________________________________________________
02/28/2021 07:53:46 VERBOSE upscale_64_0_conv2d_conv2d ( (None, 32, 32, 256) 295168
02/28/2021 07:53:46 VERBOSE _________________________________________________________________
02/28/2021 07:53:46 VERBOSE upscale_64_0_conv2d_leakyrel (None, 32, 32, 256) 0
02/28/2021 07:53:46 VERBOSE _________________________________________________________________
02/28/2021 07:53:46 VERBOSE upscale_64_0_pixelshuffler ( (None, 64, 64, 64) 0
02/28/2021 07:53:46 VERBOSE _________________________________________________________________
02/28/2021 07:53:46 VERBOSE face_out_a_conv2d (Conv2D) (None, 64, 64, 3) 4803
02/28/2021 07:53:46 VERBOSE _________________________________________________________________
02/28/2021 07:53:46 VERBOSE face_out_a (Activation) (None, 64, 64, 3) 0
02/28/2021 07:53:46 VERBOSE =================================================================
02/28/2021 07:53:46 VERBOSE Total params: 6,199,747
02/28/2021 07:53:46 VERBOSE Trainable params: 6,199,747
02/28/2021 07:53:46 VERBOSE Non-trainable params: 0
02/28/2021 07:53:46 VERBOSE _________________________________________________________________
02/28/2021 07:53:46 VERBOSE _________________________________________________________________
02/28/2021 07:53:46 VERBOSE Layer (type) Output Shape Param #
02/28/2021 07:53:46 VERBOSE =================================================================
02/28/2021 07:53:46 VERBOSE input_3 (InputLayer) (None, 8, 8, 512) 0
02/28/2021 07:53:46 VERBOSE _________________________________________________________________
02/28/2021 07:53:46 VERBOSE upscale_256_1_conv2d_conv2d (None, 8, 8, 1024) 4719616
02/28/2021 07:53:46 VERBOSE _________________________________________________________________
02/28/2021 07:53:46 VERBOSE upscale_256_1_conv2d_leakyre (None, 8, 8, 1024) 0
02/28/2021 07:53:46 VERBOSE _________________________________________________________________
02/28/2021 07:53:46 VERBOSE upscale_256_1_pixelshuffler (None, 16, 16, 256) 0
02/28/2021 07:53:46 VERBOSE _________________________________________________________________
02/28/2021 07:53:46 VERBOSE upscale_128_1_conv2d_conv2d (None, 16, 16, 512) 1180160
02/28/2021 07:53:46 VERBOSE _________________________________________________________________
02/28/2021 07:53:46 VERBOSE upscale_128_1_conv2d_leakyre (None, 16, 16, 512) 0
02/28/2021 07:53:46 VERBOSE _________________________________________________________________
02/28/2021 07:53:46 VERBOSE upscale_128_1_pixelshuffler (None, 32, 32, 128) 0
02/28/2021 07:53:46 VERBOSE _________________________________________________________________
02/28/2021 07:53:46 VERBOSE upscale_64_1_conv2d_conv2d ( (None, 32, 32, 256) 295168
02/28/2021 07:53:46 VERBOSE _________________________________________________________________
02/28/2021 07:53:46 VERBOSE upscale_64_1_conv2d_leakyrel (None, 32, 32, 256) 0
02/28/2021 07:53:46 VERBOSE _________________________________________________________________
02/28/2021 07:53:46 VERBOSE upscale_64_1_pixelshuffler ( (None, 64, 64, 64) 0
02/28/2021 07:53:46 VERBOSE _________________________________________________________________
02/28/2021 07:53:46 VERBOSE face_out_b_conv2d (Conv2D) (None, 64, 64, 3) 4803
02/28/2021 07:53:46 VERBOSE _________________________________________________________________
02/28/2021 07:53:46 VERBOSE face_out_b (Activation) (None, 64, 64, 3) 0
02/28/2021 07:53:46 VERBOSE =================================================================
02/28/2021 07:53:46 VERBOSE Total params: 6,199,747
02/28/2021 07:53:46 VERBOSE Trainable params: 6,199,747
02/28/2021 07:53:46 VERBOSE Non-trainable params: 0
02/28/2021 07:53:46 VERBOSE _________________________________________________________________
02/28/2021 07:53:46 INFO Loading Trainer from Original plugin...
02/28/2021 07:53:46 VERBOSE Loading config: 'C:\Users\faceswap_amd\config\train.ini'
Reading training images (A): 0%| | 0/1738 [00:00<?, ?it/s]
Reading training images (A): 0%| | 1/1738 [00:00<03:21, 8.63it/s]
Reading training images (A): 11%|# | 189/1738 [00:00<02:05, 12.30it/s]
Reading training images (A): 19%|#9 | 337/1738 [00:00<01:20, 17.51it/s]
Reading training images (A): 29%|##8 | 496/1738 [00:00<00:49, 24.89it/s]
Reading training images (A): 37%|###6 | 642/1738 [00:00<00:31, 35.18it/s]
Reading training images (A): 44%|####4 | 773/1738 [00:00<00:19, 49.69it/s]
Reading training images (A): 53%|#####3 | 928/1738 [00:00<00:11, 70.02it/s]
Reading training images (A): 63%|######2 | 1088/1738 [00:00<00:06, 98.19it/s]
Reading training images (A): 71%|#######1 | 1242/1738 [00:00<00:03, 136.52it/s]
Reading training images (A): 80%|######## | 1399/1738 [00:01<00:01, 188.01it/s]
Reading training images (A): 89%|########8 | 1545/1738 [00:01<00:00, 247.59it/s]
Reading training images (A): 98%|#########7| 1696/1738 [00:01<00:00, 327.14it/s]
Reading training images (B): 0%| | 0/2389 [00:00<?, ?it/s]
Reading training images (B): 6%|5 | 133/2389 [00:00<00:01, 1152.62it/s]
Reading training images (B): 12%|#2 | 291/2389 [00:00<00:01, 1253.63it/s]
Reading training images (B): 19%|#8 | 444/2389 [00:00<00:01, 1324.55it/s]
Reading training images (B): 23%|##2 | 540/2389 [00:00<00:01, 1122.78it/s]
Reading training images (B): 28%|##8 | 675/2389 [00:00<00:01, 1181.67it/s]
Reading training images (B): 34%|###4 | 822/2389 [00:00<00:01, 1254.71it/s]
Reading training images (B): 40%|#### | 966/2389 [00:00<00:01, 1304.16it/s]
Reading training images (B): 47%|####7 | 1125/2389 [00:00<00:00, 1377.57it/s]
Reading training images (B): 54%|#####3 | 1290/2389 [00:00<00:00, 1448.36it/s]
Reading training images (B): 60%|###### | 1435/2389 [00:01<00:00, 1199.71it/s]
Reading training images (B): 67%|######6 | 1589/2389 [00:01<00:00, 1284.05it/s]
Reading training images (B): 73%|#######2 | 1734/2389 [00:01<00:00, 1328.74it/s]
Reading training images (B): 78%|#######8 | 1875/2389 [00:01<00:00, 1351.10it/s]
Reading training images (B): 85%|########4 | 2028/2389 [00:01<00:00, 1399.21it/s]
Reading training images (B): 91%|#########1| 2179/2389 [00:01<00:00, 1429.64it/s]
Reading training images (B): 97%|#########7| 2325/2389 [00:01<00:00, 1437.50it/s]
02/28/2021 07:53:50 CRITICAL Error caught! Exiting...
02/28/2021 07:53:50 ERROR Caught exception in thread: '_training_0'
02/28/2021 07:53:50 ERROR You have selected the Mask Type 'extended' in your training configuration options but at least one face does not have this mask type stored for it.
02/28/2021 07:53:50 ERROR You should select a mask type that exists within your face data, or generate the required masks with the Mask Tool.
02/28/2021 07:53:50 ERROR 0 faces in side A do not contain the mask 'extended'. Available masks: {'extended', 'components'}
02/28/2021 07:53:50 ERROR 149 faces in side B do not contain the mask 'extended'. Available masks: {'extended', 'vgg-obstructed', 'components'}
Process exited.