Crash: Inappropriate ioctl for device

Training your model
Forum rules
Read the FAQs and search the forum before posting a new topic.

Please mark any answers that fixed your problems so others can find the solutions.
Locked
User avatar
blackomen
Posts: 9
Joined: Fri Feb 21, 2020 2:14 pm
Has thanked: 2 times
Been thanked: 1 time

Crash: Inappropriate ioctl for device

Post by blackomen »

Here's what I'm getting when I try to run this on a cloud service:

02/23/2020 13:43:13 INFO Loading data, this may take a while...
02/23/2020 13:43:13 INFO Loading Model from Dfaker plugin...
02/23/2020 13:43:13 INFO No existing state file found. Generating.
02/23/2020 13:43:14 ERROR Got Exception on main handler:
Traceback (most recent call last):
File "/home/paperspace/faceswap/lib/cli.py", line 128, in execute_script
process.process()
File "/home/paperspace/faceswap/scripts/train.py", line 157, in process
err = self._monitor(thread)
File "/home/paperspace/faceswap/scripts/train.py", line 343, in _monitor
keypress = KBHit(is_gui=self._args.redirect_gui)
File "/home/paperspace/faceswap/lib/keypress.py", line 42, in __init__
self.new_term = termios.tcgetattr(self.file_desc)
termios.error: (25, 'Inappropriate ioctl for device')



My crash log is attached
Attachments
crash_report.2020.02.23.134314435210.log
(33.8 KiB) Downloaded 29 times

User avatar
torzdf
Posts: 650
Joined: Fri Jul 12, 2019 12:53 am
Answers: 95
Has thanked: 17 times
Been thanked: 131 times

Re: Crash: Inappropriate ioctl for device

Post by torzdf »

I assume you are running this in a Jupyter Notebook of some kind?

We have a function that monitors for multiple key presses. As Faceswap is multi-platform, we rely on a third party lib to monitor the terminal for key presses. It would appear that this doesn't play nicely with PaperSpace.

This is unlikely to be fixed (I cannot test in the environment you are in), so you may need to hack around in the code.
My word is final

User avatar
blackomen
Posts: 9
Joined: Fri Feb 21, 2020 2:14 pm
Has thanked: 2 times
Been thanked: 1 time

Re: Crash: Inappropriate ioctl for device

Post by blackomen »

No, I was trying to run it from the command line from my cloud service (after failing to get the GUI to work via VNC.) I used the Generate button on my local computer's copy to get the commands and uploaded my faces and alignments to my cloud service and adjusted the paths in the generated command. I then ran try to run the generated command with a "nohup" in front and "> train.log &" after to get it to run indefinitely and to log to a file.

User avatar
torzdf
Posts: 650
Joined: Fri Jul 12, 2019 12:53 am
Answers: 95
Has thanked: 17 times
Been thanked: 131 times

Re: Crash: Inappropriate ioctl for device

Post by torzdf »

It's entirely possible that nohup is hiding the terminal from Faceswap, hence why you get that error (no terminal = nothing to monitor = error).

I suggest installing screen, and launching faceswap there (so you can happily leave it running and just log back into the session to check progress).

https://help.ubuntu.com/community/Screen

Also, Faceswap automatically logs to file, so redirecting stdout is not necessary.
My word is final

User avatar
blackomen
Posts: 9
Joined: Fri Feb 21, 2020 2:14 pm
Has thanked: 2 times
Been thanked: 1 time

Re: Crash: Inappropriate ioctl for device

Post by blackomen »

Yeah, it works when ran normally from the terminal alright. However, I don't like the idea that losing my SSH connection will terminate my training. I'll look into maybe running this on Jupyter instead (esp since I got it to work on colab but it'll be nice to be able to train multiple models simultaneously on colab, my cloud provider, and my local machine.)

User avatar
torzdf
Posts: 650
Joined: Fri Jul 12, 2019 12:53 am
Answers: 95
Has thanked: 17 times
Been thanked: 131 times

Re: Crash: Inappropriate ioctl for device

Post by torzdf »

blackomen wrote:
Tue Feb 25, 2020 2:17 am
I don't like the idea that losing my SSH connection will terminate my training.
This is precisely why I suggested Screen. You should look into it... it's entire existence is so you can connect and disconnect to terminals without terminating the running applications.
My word is final

Locked