Re: gmake and ccache conspiring together in creating gremlins

[Paul Smith]

On Mon, 2021-02-08 at 10:43 +0000, Edward Welbourne wrote:
> Sounds to me like that's a bug: when the descriptors are closed, the
> part of MAKEFLAGS that claims they're make's jobserver file
> descriptors should be removed, since that's when the claim stops
> being true.

I believe there have been other similar issues reported recently.

Certainly fixing MAKEFLAGS when we run without jobserver available is
something that could be done.

There is a loss of debugging information if we make this change: today
make can detect if it was invoked in a way that _should_ expect to
receive a jobserver context, but _didn't_ receive that context.  That
is, if make sees that jobserver-auth is set but it can't open the
jobserver pipes it can warn the user that most likely there's a problem
in their environment or with the setup of their makefiles.

Without this warning there's no way to know when this situation occurs.
 It's easy to create a situation where every sub-make will create its
own completely unique jobserver domain.  So you start the top make with
-j4 and run 4 sub-makes; if you do it wrong then each of 4 sub-makes
could create a new jobserver domain, and now you're running 16 jobs in
parallel instead of 4... there's no way for make to warn you about this
situation.


Another option that I'm considering is moving away from anonymous pipes
and switching to either named pipes or named semaphores instead (I'm
not sure if one or the other is preferred WRT portability).  If I did
that then all of this hullabaloo around open/closed file descriptors,
inherited FDs opened blocking vs. non-blocking, and "passing through"
jobserver access across non-make boundaries would go away.

I liked the original implementation for these reasons:
 * It is very generic: pretty much every system supports simple pipes.
    However, in the end only POSIX systems are using anonymous pipes
   anyway (Windows jobserver already uses Windows named semaphores).
 * It is very easy to manage: using named pipes means that make is
   creating context on the filesystem that it needs to manage and clean
   up, which it otherwise never does; we need to worry about
   permissions, etc.  Anonymous pipes just go away magically.
 * It is very safe: it's not possible for any other process to access
   the pipe and mess up the jobserver count, unless it was invoked by
   make in a "sub-make context".

But, it is difficult to use in some subtle ways as we've seen.

A change would also mean that the format of the --jobserver-auth flag
would change: if the value provided were the current 2 numbers then the
old-school anonymous pipe process would be used.  If it were a path,
then we'd assume it was a named pipe (or named semaphore).

Other tools like LTO etc. that look for jobserver-auth would,
hopefully, be able to manage this.  I tried to be clear about the
accepted formats and behaviors in the GNU make documentation; hopefully
developers are handling incorrect formats properly.