Feedback and suggestions after using faceswap for a couple months

Discussions about research, Faceswapping and things that don't fit in the other categories here.

User avatar
Posts: 95
Joined: Sat Jan 18, 2020 6:32 pm
Has thanked: 13 times
Been thanked: 34 times

Feedback and suggestions after using faceswap for a couple months

Post by cosmico »

Suggestions tweaks
-On the analysis tab during training, automatically start with the most recent training session on top, or at the very least an option to turn this on so essentially session number descending rather than ascending, or the option to make this default. Example: If I've done 90 sessions, it always starts at session 1 and I have to scroll all the way to 90 to see how my most recent sessions have been going.

-For the graphs, or the ability to set that as default smoothing value. So that I can choose to have it default to .99 rather than have it start at .9 and change it every time

-ability to Adjust the preview size during the conversion process, So I can see one large preview instead of 6 medium pictures. If this is already an option, perhaps an easier way to find this setting

-Big one: When you select train or convert or extract at the very bottom, the button remains "semi-selected" which has caused problems. For example, you get everything ready to train before you leave for a while, you press train, and then if you use your computer just a little bit more before leaving to do your thing, you might accidentally press spacebar
while the program is open, and since those buttons turn into "stop", you stopped the process with out realizing it, and you come back much later to learn that no training was made over that time. this issue is somewhat related to the fact that alot of times, it can take a while after pressing the stop command before it truly stops perhaps when you click train or convert, it does keep it semi-selected so you don't press spacebar or enter as you switch to another program telling it to stop what its doing.

-you can open up multiple analysis summary statistics graphs of the same exact session if you click on it to many times. You don't need 10, or even 2 graphs of the same exact session since they both show the exact same information. (sometimes you accidentally open more than one). So if that could be limited to one it would be nice.

-When you switch projects, the plugins like coverage and learning rate does not change. perhaps on one project, im doing a low learning rate with a small coverage but my next project is the opposite, when i change projects
the plugins should reflect that.

-When using graph tab, holding your mouse over a certain part of the line should tell you its linear value. this could be helpful because when doing long session, even with the line smoothed to .99 the peaks and valleys will override the grid that tells us where it is. I can see that its somewhere between .35 and .3 but I have no idea if its near .31 or .33. If I can just
put my pointer on the line, it could tell me its value at that exact spot.

-When you edit the alignments like removing faces from the video, its creates the new alignment file plus a backup with the same name but with some numbers added to it. It would be nice if it had the word backup after the original file name then the numbers, just for ease of use and identification for beginners

-When using the alignment tool to remove faces, at the bottom in the gui's terminal, it says all the information but the very last thing it states after several empty lines is how many faces were renamed. it would be niced if it said how many alignments were removed as well right next to instead of up being up like 6 lines in the middle of all
that text

-The gui for multiple faces/removing some faces but not all in a frame with multiple faces should be redesigned for easy understanding. it should be explain like I'm 5 level, but it isn't. Its considerably more complex than the rest of deepfaking process. I'll be honest I still haven't figured out how to properly use it.

-I'm not sure what the giant row on the bottom of the program is called, but the sectioned box where it says things like
02/27/2020 13:32:04 INFO Successfully set up for PlaidML
Using plaidml.keras.backend backend.
02/27/2020 13:32:08 INFO Model A Directory: [DIRECTORY NAME AND PATHWAY]
When you start a new training or conversion process, it eliminates all that data from that box. It would be nice if we could see the last session or two before the current one being worked on. Mainly for the type of situation where you notice it stopped training and without even thinking about it, you click train again, and then immediately realized it might have stopped due to an error, and thus you can scroll up and read/copy-paste what

-in the tools>alignments tab, under the jobs buttons; do something with the remove-faces/remove-frames situation. I understand that the names are all in alphabetical order but they have incredibly similar names, and they're functions aren't insanely different. As a result there pretty easy to mistake the two. For the longest time, I couldn't remove the alignments from my alignment file because I had remove frames instead of remove faces selected. Maybe you could rename it to remove unwanted alignments or something don't mistake them. Its a pretty easy mistake to make for newbies and it can even happen if you know what you are doing. Honestly even if you know what you are doing,
I feel the name of those tools are a little confusing. While they mean different things, its so easy to think of a "faces", "frames", and the thumbnails we see extracted, to by synonymous with another in this application.
for example of im deepfaking trump and I see that I extracted some pictures of pence who was in the background, and thus I have alignments of pence, its so easy to just to think "ok now lets remove all the frames of pence" which would incorrectly lead me down the path of remove frames tools rather than remove faces tool.

-In general I felt all the tools>alignments>jobs could use a better description both in gui and on the guide. They are noticeably less detailed compared to other parts of the guide. Proper understanding (heck even over-explanation) is key to user success and satisfaction and keeping less unneeded questions out of the forum.

-no longer relevant but I''ll still mention it: When scrolling through the timelapse files, The images in a time lapse picture all randomized. Can this be made so that they are all stationary? the benefit in this is that you can quickly scroll through you're timelapse pictures to see how the faces have changed over time, which is a great visual
way to see all the progress thats being made. you can still do this technically when the faces are in a randomized spot, but its so slow trying to find the exact picture from two randomized collages of pictures, that you've probably forgotten the small differences from the previous picture by the time you find it. I've noticed faceswap already does what I'm asking, ....but only on one of the previews. like usually the one in the bottom right corner will stay the same throughout all the timelapse pictures while the rest of them mix around in a crazy order. do this but to all of them. Since I wrote all this, I figure I just mention that my most recent deepfake didn't do this for the very first time. I don't know why or how I managed to do that.
^^^Update^^ not long after I wrote this, I uninstalled and reinstalled for different reasons it stopped doing that issue. Now there all in the same place when i look through timelapse photos. I dont know if you did something to fix this with in the past 2 months or if something just went bad in my initial installation and times experimenting with the program.

-Better explanation in program/gui (rather than guide, although better and better explanations in guides are always appreciated) of what loss A and B correlate too.
Does a really low Loss B mean the program thinks it really is understanding Face B? or is this slightly backwards and somehow correlates to Face A's understand in the
process of swapping B onto A.

-GUI error when you adjust the size of the column on the left: the column on the left containing extract-train-convert-tools etc., when you adjust it to make it really skinny so you can have a very large view of the training preview or conversion process, the vertical scrolling slide for going up and down that column disappears. Apparently there is a minimum
width required to actually be able to go up and down that column. So if you make it to skinny but you want to scroll down to view your global options, you can't unless you make that column not skinny first and then the scroller reappears.

-Why is sort its on tab but also under tools>alignments>jobs ? While I haven't made a mistake here, it seems like a potential newbie pitfall, and could be fixed just by giving it a new name that describes the same basic thing, but isn't a taken word like sort. -Unless it is the exact same "sort" as the one in a separate tab sort and just hasn't been removed. in which case it would be nice to have it removed.

Suggestions for the guide specifically
-For the guide: The guide is probably the single most important thing for user success and satisfaction. So I would love constant updates and improvements, even the
tiniest things help. I'd rather have a long guide than a short one.

-For the guide: There is already an images of what the different face coverages will result in, but it would definitely be improved if there were gifs showing a the same exact deepfake done 3 times, and these are the final results with 3 different coverage levels just as an example.

Suggestions features for faceswap
-A test feature that once you both source and to-be-swapped images ready to go, you could run, and could give you results of what training settings would be best for the trainer you selected. Idk it could run a couple times on every batch size increasing by 2 each time, giving you a basic graph of what what point you'll achieve max EG/s, max iterations, max...any other info you might want to know.

-Adding a prediction loss trend line on the analysis summary statistics graph that goes beyond where you currently are. The current estimate line stops at where you are
currently at in training and it would be nice if it continued out indefinitely or at least to the next 100,000 iterations. I know alot of variables come into play that it cant predict and it wouldnt be super accurate, but it still would be a nice feature if I could roughly tell that with in the next 20,000 iterations my loss value would drop by .1 or if I would hit .1 loss value after 100,000 iterations from my current training spot

-A preview for the sharpness scaling setting while converting, even if its just a preset picture and not a frame from whatever project you are working, just so you can see that...say in your specific situation, having a threshold of 300 is fine, but making your radius 2 as opposed to .5 would just produce weird results. Im thinking something similar to the
coverage picture in the guide that show you what the results of what coverage is being used. Doing this would allow you to see how the deepfake would look without having to start a conversion and then looking at the previews.

-having a naming option for when you convert a video and it saves it. That way if you havent renamed or moved your last attempt at conversion, it won't overwrite it. While a explorer window popping up allowing to name and save it in a certain place would be cool, even just the program automatically naming the video based off how many eg/s or
iterations it went through would be fine. Even simpler than that would be adding a number after the file so that when you convert, you get trump_converted.mp4, but when you try a diffrent sharpness setting and convert again, the new file is trump_converted(1).mp4 and then trump_converted(2).mp4, rather than overwriting the original converted file

  • A button in the gui near the timelapse that opens up the selected timelapse folder in an explorer window would be nice, so you can quickly have an explorer window on your timelapse folder allowing you to view the timelapses rather than having to go all thr would though you computer files to wherever you actually placed that timelapse folder.

-Just a simple feature that tells us when an updated version of faceswap is available, definitely dont make it annoying like so many software's update announcement systems