1920x1080 video and output size

The Extraction process failing on you, and you aren't getting an error back with clear instructions? Tell us about it here


Forum rules

Read the FAQs and search the forum before posting a new topic.

This forum is for reporting errors with the Extraction process. If you want to get tips, or better understand the Extract process, then you should look in the Extract Discussion forum.

Please mark any answers that fixed your problems so others can find the solutions.

Locked
User avatar
FaceRunner
Posts: 3
Joined: Fri Jan 06, 2023 5:47 pm

1920x1080 video and output size

Post by FaceRunner »

Hi all.

First, thanks for everyone's input and this software etc. It's great that people do these things...

equip: Nvidia RTX 3070 w/8GB RAM, i9 12900K CPU, 32GB RAM, Linux

I searched as much as I can and have read the guides but I'm just not able to formulate an answer.

I am using HD 1920x1080 video that I am extracting. The Extract tab\output size is defaulted at 512. After more readings, I see this may be something I need to change.

I was able to train Phaze-A for a couple of days at 128, no issues, but face was just blurry etc. So my research told me I may need to bump the train to 256... then it looked better. So I tried 512... looked even better!

But, the Training (Phaze-A, all settings default except output size is 512) keeps exiting with errors after about 30 minutes (I will post these later if I can't get it to work).

I can reduce settings and get no errors, so I am attributing errors and exits of the app to the PC running out of resources.

I would just test it but it takes days just to get viewable results so I thought asking would be quicker.

So, I reduced Phaze-A to 256 (so far no exits) but I wonder if I should reduce the Extract output to 256 to match.

What is weird is I have another computer with the same setup except it uses an Nvidia GTX 1070 w/8GB ram, and an i7 CPU, 32GB RAM, Linux and Phaze-A has been training for 3 hours at 512 output (all other settings default), 512 Extract, no issues so far.

Once I Convert to video (FFMPEG, .MOV) will an Extract size of 256 be sufficient for this video or should I bump it back to 512 or even 1024?

TIA

User avatar
torzdf
Posts: 2649
Joined: Fri Jul 12, 2019 12:53 am
Answers: 159
Has thanked: 128 times
Been thanked: 622 times

Re: 1920x1080 video and output size

Post by torzdf »

Ok.... there's a lot to unpack here....

FaceRunner wrote: Fri Jan 06, 2023 6:13 pm

I was able to train Phaze-A for a couple of days at 128, no issues, but face was just blurry etc. So my research told me I may need to bump the train to 256... then it looked better. So I tried 512... looked even better!

What does this mean? Can you specifically tell me the steps you are taking to change the size?

I can reduce settings and get no errors, so I am attributing errors and exits of the app to the PC running out of resources.

Very likely. It shouldn't be able to run out of resources significantly into a train, but the reality is this does happen somewhere, which I guess is some slight resource leak somewhere in Tensorflow

So, I reduced Phaze-A to 256 (so far no exits) but I wonder if I should reduce the Extract output to 256 to match.

No leave it at default. Only really need to adjust the extract size when going above about 384px. The model is trained on a 'sub-crop' of the training image (based on centering and coverage) so extract should be larger than model output side (see here: https://github.com/deepfakes/faceswap/pull/1095)

What is weird is I have another computer with the same setup except it uses an Nvidia GTX 1070 w/8GB ram, and an i7 CPU, 32GB RAM, Linux and Phaze-A has been training for 3 hours at 512 output (all other settings default), 512 Extract, no issues so far.

This sounds... unlikely, but I would need to understand the steps you are taking to change the size better

Once I Convert to video (FFMPEG, .MOV) will an Extract size of 256 be sufficient for this video or should I bump it back to 512 or even 1024?
{/quote]
That is hard to say, and will come from experience. If this is your first swap, I would start with the lower res whilst you learn the process. Everyone's first swap is bad. Best to get there in the shortest amount of time ;)

My word is final

User avatar
FaceRunner
Posts: 3
Joined: Fri Jan 06, 2023 5:47 pm

Re: 1920x1080 video and output size

Post by FaceRunner »

torzdf wrote: Sat Jan 07, 2023 12:00 pm
FaceRunner wrote: Fri Jan 06, 2023 6:13 pm

I was able to train Phaze-A for a couple of days at 128, no issues, but face was just blurry etc. So my research told me I may need to bump the train to 256... then it looked better. So I tried 512... looked even better!

What does this mean? Can you specifically tell me the steps you are taking to change the size?

-- In Settings\Configure Settings\Train\Model\Phaze-A I set the Output Size. This is what I am referring to. Default is 128, fine, blurry, no exits. I tried 256, better, no exits. 512 better but exits after a time. I put it back to 256 and so far on 256 it has rendered for 233K iterations and 24+ hours. This is the only setting here that I adjusted. This is the i9 PC

So, I reduced Phaze-A to 256 (so far no exits) but I wonder if I should reduce the Extract output to 256 to match.

No leave it at default. Only really need to adjust the extract size when going above about 384px. The model is trained on a 'sub-crop' of the training image (based on centering and coverage) so extract should be larger than model output side (see here: https://github.com/deepfakes/faceswap/pull/1095)

-- Ok so here, Extract tab\Output size defaults to 512. I reduced this to 256 because power PC i9 was exiting the train and I figured this may be a cause. I will put this back to 512 to see what happens. So to be clear, Phaze-A output size is now at 256 and I will set extract output size to 512, the default.

What is weird is I have another computer with the same setup except it uses an Nvidia GTX 1070 w/8GB ram, and an i7 CPU, 32GB RAM, Linux and Phaze-A has been training for 3 hours at 512 output (all other settings default), 512 Extract, no issues so far.

This sounds... unlikely, but I would need to understand the steps you are taking to change the size better

-- Per above, the two sizes I am referring to are Phaze-A Output size and Extract tab Output size. These are the only two settings I have changed. My guess is the i9 is a significantly different CPU architecture than the i7. But, both power PCs have the same size system RAM, GPU RAM and operating system (Linux). We can dig into specific errors if you like and I can send the crash reports.

Once I Convert to video (FFMPEG, .MOV) will an Extract size of 256 be sufficient for this video or should I bump it back to 512 or even 1024?

That is hard to say, and will come from experience. If this is your first swap, I would start with the lower res whilst you learn the process. Everyone's first swap is bad. Best to get there in the shortest amount of time ;)

-- So the app has exited out on the less powerful PC, i7. Extract was at 512 and Phaze-A is at 512. It ran a good 20+ hours and 135K iterations. I expected it to do so though because it's about 4X slower than the big PC and just took longer to crash :D . I did restart it but it stops every few hours now. So I'll prob reduce the Phaze-A to 256.

Oh I wanted to add: for power PC i9 I am using a Training batch size of 8. For the lesser power PC i7, I am using Training batch size 2.

p.s. They are both coming out well after hundreds of Ks of iterations!

Thanks for the help!

User avatar
torzdf
Posts: 2649
Joined: Fri Jul 12, 2019 12:53 am
Answers: 159
Has thanked: 128 times
Been thanked: 622 times

Re: 1920x1080 video and output size

Post by torzdf »

FaceRunner wrote: Sat Jan 07, 2023 5:53 pm

-- In Settings\Configure Settings\Train\Model\Phaze-A I set the Output Size. This is what I am referring to. Default is 128, fine, blurry, no exits. I tried 256, better, no exits. 512 better but exits after a time. I put it back to 256 and so far on 256 it has rendered for 233K iterations and 24+ hours. This is the only setting here that I adjusted. This is the i9 PC

No, I get that. Sorry, I wasn't clear.... What i'm getting at. When you change output size of the model, do you delete the original model folder and/or start training pointing at a new folder or do you leave the model folder the same after changing the output size settings? How you do this will help me better understand what is going on.

My word is final

User avatar
FaceRunner
Posts: 3
Joined: Fri Jan 06, 2023 5:47 pm

Re: 1920x1080 video and output size

Post by FaceRunner »

torzdf wrote: Sun Jan 08, 2023 2:22 pm
FaceRunner wrote: Sat Jan 07, 2023 5:53 pm

-- In Settings\Configure Settings\Train\Model\Phaze-A I set the Output Size. This is what I am referring to. Default is 128, fine, blurry, no exits. I tried 256, better, no exits. 512 better but exits after a time. I put it back to 256 and so far on 256 it has rendered for 233K iterations and 24+ hours. This is the only setting here that I adjusted. This is the i9 PC

No, I get that. Sorry, I wasn't clear.... What i'm getting at. When you change output size of the model, do you delete the original model folder and/or start training pointing at a new folder or do you leave the model folder the same after changing the output size settings? How you do this will help me better understand what is going on.

Ahhh. I delete everything and start over. I don't know what the repercussions of leaving anything from the first train would be so I play it safe and just start over. I delete my extract folders, model, timelapse, alignments etc. All of it. It's no biggie while I'm learning this to start over.

So far I am still getting fairly blurry renders even after 2 days and 350K iterations. I know it could take a week and a million iterations but I'm also afraid of over-training and am not sure when to switch the 'Augmentation' to 'No Warp' (currently I always have 'Warp to Landmarks' enabled). I know when it shows pretty clear previews to switch, but that seems highly subjective...

Once I feel like I can change to 'No Warp', how long should I train using 'No Warp'?

Locked