Ah, I read over that bit. My bad. OK, /dev/tty won’t work if the process has no controlling terminal, so the open() will fail. As I said, a process inherits its controlling terminal (or lack thereof), from its parent, and simply opening a terminal device (as Xcode does when it sends the debugged process’ output to a pseudo-terminal), won’t automatically make it a controlling terminal. Xcode could execute the necessary ioctl() to make it a controlling terminal. I don’t know if its failure to do this is simply an oversight on Apple’s part or a deliberate decision to avoid some implications of having a controlling terminal that I don’t know about. Controlling terminals generate job control signals (SIGTSTP, SIGTTIN, SIGTTOU), and maybe Apple has some reason for wanting to avoid those.

If Xcode is directing stdout to a pty, then you should be able to insert an ioctl(1. TIOCSCTTY, 0) call in your code in order to make it a controlling terminal. If the process already has a controlling terminal, or if stdout is not a tty device, then this call will fail harmlessly.

So what happens if you launch Xcode from the terminal?

Given that Xcode has the ability to send stdout to a pty but hasTerminal() doesn’t seem to be picking up on that fact, I can think of two possibilities: 1) Xcode connects stdout to the pty but not stdin, and hasTerminal() only checks stdin; or 2) hasTerminal() checks for a controlling terminal. Feel free to continue your assembly traipsing to figure out which it is. :-)

In fact it is just as simple as I described in the entry: hasTerminal checks only for “/dev/tty”. It doesn’t check for file descriptors or anything. Just calls open(“/dev/tty”, 2).

It’s been a while since I looked under the hood, but here goes…

/dev/tty is a pseudo-device that the kernel guarantees always refers to a process’s controlling terminal, if it has one. A process inherits its controlling terminal from its parent, but can disassociate itself from its controlling terminal (in which case its children will also have no controlling terminal). As far as I can tell, GUI programs have no controlling terminal if they aren’t run from the command line. XCode can allocate and open a pty device (pseudo-terminal, see pty4. man page), for a process it is debugging. However, just opening a tty device, pseudo or otherwise, does not automatically make that device a controlling terminal. It takes a specific ioctl() call to make it a controlling terminal, and thereby enabling /dev/tty for that process.

On the other hand, there is no requirement that a terminal be a *controlling* terminal. A controlling terminal is important for Unix-style job control (foreground and background processes, terminal signals, etc.), but not for plain old terminal I/O. I don’t know what hasTerminal() does under the hood, but the very similar C standard library function isatty() takes a file descriptor as an argument and tells you whether or not it refers to a terminal device, irrespective of whether or not it is the process’ controlling terminal. It does this by executing an ioctl() that only works for terminal devices (see the tty.4 man page for a list; isatty() uses a nondestructive call like TIOCGETA or TIOCGETD).

In either case, starting a process from the command line both will give it a controlling terminal and will have the stdin/out/err file descriptors open on that terminal (the latter assuming no I/O redirection is done). Given that Xcode has the ability to send stdout to a pty but hasTerminal() doesn’t seem to be picking up on that fact, I can think of two possibilities: 1) Xcode connects stdout to the pty but not stdin, and hasTerminal() only checks stdin; or 2) hasTerminal() checks for a controlling terminal. Feel free to continue your assembly traipsing to figure out which it is. :-)

I don’t know enough about how “/dev/tty” gets hooked up to the process’s pseudo ttys to know who is really at fault here.

As much as people love to bitch about old school Carbon programming, at least Carbon had the ever so useful OSErr (and later OSStatus) types to inform the programmer what went wrong with a OS function call. While the errors returned from old Carbon APIs were often cryptic, at least they would point you in the right direction. With the CGBitmapContextCreate problem I was making all sorts of wild guesses before I found the Q&A article. Returning ‘paramErr’ from the function (and returning the CGContextRef as a indirect parameter) would have saved me a little bit of time.