I made this project year ago, and the result was disappointing for me, but learned lots of lessons
The second half becomes better but the resolution becomes a thing at that point. (faces in the glasses worked pretty good)
The original had extra animations before, between and after but had cut that out because its in dutch and not relevant.
A client who we had worked before with held a event in the matrix theme, we would film the event.
So i came up with the idea as a little extra to put in some deepfake of the client in place of Neo. (just as a fun thing to put face swap in use for actual project).
So would be in the opening video of the event.
This was a trial and error process, with a couple of things really hindering progress.
Extracting faces that are heavy shadowed really messes with the training, when you try to brighten the shaded parts it just brings up the noise like crazy and also interferes with the training. Masking becomes shaky du to the heavy grain in the film.
Heavy shadows makes it really hard to integrate the face swap over the original. (in the final resolve i placed gradients over the face for better blend)
The matrix also has really heavy film grain so in the end i denoised the film first, extracted the faces. after the swap i renoised the total again.
Trained this model as dyn1024 with some home made changes (don't know if it helped or hurt in the end)
But the training time is horrible with these big high res models and the time in the end was just to short to really properly trained.
Result is after 250000 It's with a batch size of 4. But even that took 2 or 3 weeks, with al the trial and error of getting good training faces and stuff not included.
Also for a while the client was indecisive what part of the film he wanted for the deepfake so in the beginning trained with way more that the end scene so that didn't help as well
Now i can look back and smile because it was a cool project and if ever again would do so many things different, live and learn.
But still its a uncommon scene for deepfake, so thought lets just share 
