PlashGlibc calls not async signal safe
A number of calls are defined by POSIX as being "async signal safe", which means safe to call from signal handlers. See this list. It includes open() and other calls which in PlashGlibc are implemented via RPCs. Currently these calls are not safe to re-enter from signal handlers and doing so could corrupt PlashGlibc's data structures or send mixed-up messages over the socket connection to the ServerProcess. I noticed this problem while looking at PlashIssues/ForkDeadlock.
It could be difficult to implement this.
There are two halves to this:
Dealing with re-entering PlashGlibc RPC functions.
It should be possible to defer running signal handlers until the end of the RPC call. This might be expensive if it requires making additional system calls. In principle that should not be necessary, because it should be possible to queue the signal handlers entirely in process (see Remembering a Signal in the glibc manual).
- Alternatively, we could detect re-entering a call and return an error. That probably does not meet the requirements of the standard but it would be better than crashing.
- Using only async-signal-safe calls when doing RPCs. This code would want to use malloc(). We might need to use a custom memory allocator for this to work.
Is there a way to find out whether we are currently inside a signal handler? Probably not. It is possible to jump out of signal handlers using longjmp(). Signal handlers can themselves be re-entered if a second signal occurs. If the process is set to use a separate signal stack, we could detect whether we are in a signal handler based on the stack pointer; but the process won't always be using a separate stack for signal handling.
Does the Hurd have any difficulties implementing this? What about Cygwin? Did Ostia handle this?