So, I'm experimenting with a new approach to using the cloud instances, where I run everything in the startup-script, and have an external watcher keep an eye on the instance. This way, I can automate restarting/rezoning my preemptible instances, and save money over using non-preemptible for the bigger jobs.
Unfortunately, when training runs in the startup-script, it fails with the "inappropriate ioctl for device" error, presumably because there's no terminal to speak of.
Is there either a way to turn off the interactive key mode entirely when running?
Failing that, has anyone figured out a good way to fake a terminal in such an environment as a GCE startup script?