Simple idea for improving mask erosion

Got questions or tips about the Conversion process? This is the place to discuss them.


Forum rules

Read the FAQs and search the forum before posting a new topic.

This forum is for discussing tips and understanding the process involved with Converting faces from your trained model.

If you are having issues with the Convert process not working as you would expect, then you should post in the Convert Support forum.

Please mark any answers that fixed your problems so others can find the solutions.

Locked
User avatar
martinf
Posts: 27
Joined: Thu Sep 29, 2022 7:58 pm
Been thanked: 3 times

Simple idea for improving mask erosion

Post by martinf »

Here is an idea for an improvement for the mask erosion parameters...

Human faces are not rectangular. The important features are more of a triangle or trapezoid. Having the ability to erode a mask at 'top, bottom, left and right' seems to miss some of the obvious areas of concern... the lower right and left corners.

I would suggest adding the ability to erode from the corners in addition to the edges, especially the lower right and left corners where funky things can happen a jawlines.

Doable?

As it stands now, trying to erode the jawline ends up eroding the left eyebrow as well... an undesired outcome. If the 'bottom' is eroded sufficiently to remove the jawline... there goes the mouth too. These lower two corners as targets for erosion would be great.

User avatar
torzdf
Posts: 2651
Joined: Fri Jul 12, 2019 12:53 am
Answers: 159
Has thanked: 129 times
Been thanked: 622 times

Re: Simple idea for improving mask erosion

Post by torzdf »

Sure, I may look at this in the future.

The main problem with "directional erosion" is that it may work for certain angles, but then doesn't work for other angles (e.g. eroding from the left will have a different impact if the face is looking forward to if the face is looking to the left).

The reason it was implemented was due to an unforeseen issue with full-face masking, where swapping someone with a high hairline with someone with a low hairline led to a dark shadow at the top of the head. I implemented 'top-down' erosion for that, and just added the other directions in, because I was there.

Ultimately for really good masking solutions, the mask should be output and then fixed up in something like After Effects. Sadly, any solution I implement is only ever likely to be a 'quick and dirty' fix.

With that said, I would not refuse a PR implementing this kind of thing.

My word is final

User avatar
martinf
Posts: 27
Joined: Thu Sep 29, 2022 7:58 pm
Been thanked: 3 times

Re: Simple idea for improving mask erosion

Post by martinf »

One more question...

Do the current mask erosion functions follow the pitch and yaw of the face in the final conversion? It appears as if they may only be applied to the world coordinates in the conversion.

User avatar
torzdf
Posts: 2651
Joined: Fri Jul 12, 2019 12:53 am
Answers: 159
Has thanked: 129 times
Been thanked: 622 times

Re: Simple idea for improving mask erosion

Post by torzdf »

The erosion settings are applied to the "aligned" face. IE top will always be the top of the face (even if the face is really upside down in the actual frame)

My word is final

User avatar
martinf
Posts: 27
Joined: Thu Sep 29, 2022 7:58 pm
Been thanked: 3 times

Re: Simple idea for improving mask erosion

Post by martinf »

Appreciate your quick responses, BTW. I am a professional photographer and have been using Photoshop for... oh, about 25 years so the way I think about layers and such is perhaps a bit different. The issue that you describe... forehead boundaries... might also be solved by creating a special case for the top of the mask in which the mask could be 'feathered' procedurally in the conversion and preview functions. I get that the sides and chin have to be a harder boundary, but there is that often huge chunk of facial real estate that gets that clearly demarcated boundary that is bothersome.

A second idea might be to allow for the application of an adjustable feathering feature for just the forehead. Perhaps following a slightly curved path since most hairlines are curved a bit. The user could input the percent of feathering in the conversion settings. That way, this boundary could be smoothed (feathered) as well as raised or lowered.

Just some thoughts from a non-coder who likely has no clue how difficult this would be.

Locked