Re: feature request - order only deps

[Matěj Týč]

On Ne, 2013-02-03 at 23:40 +0100, Sebastian Pipping wrote:
> On 03.02.2013 23:20, Matěj Týč wrote:
> >> If that happens how about replacing
> >>
> >>   $(MAKE) cache-foo
> >>
> >> by something like
> >>
> >>   mkdir .lock 2>/dev/null || exit 0 ; \
> >>       $(MAKE) cache-foo ; \
> >>       ret=$$?; \
> >>       rmdir .lock && exit $${ret}
> >>
> >> The idea is:
> >>
> >>  - mkdir can only succeed once
> >>
> >>  - if "$(MAKE) cache-foo" fails
> >>
> >>     1. the whole should return non-zero
> >>
> >>     2. .lock is not left laying around
> > 
> > Thank you,
> > but just by looking at it, I think that if .lock exists, 0 is returned
> > and while one make job is busy loading the cache, the others act like
> > the cache was loaded.
> > Writing the makefile and trying it confirms this assumption, so I guess
> > that it has to behave like that.
> 
> I see, so actually waiting is needed.
> 
> Have you tried modifying the call to do that?  I'd bet it's possible :-)

Well, an infinite loop checking every sec. whether the .lock directory
still exists might do the trick. However, additional check whether the
cache load was succesful would be needed and actually the directories
would have to have different names for diferent cache that could get
loaded.

> > I am beginning to think that the most elegant (from the user's point of
> > view) would be (probably optional) ignore of order-only dependencies if
> > the target exists and is more up-to-date than its ordinary dependencies.
> 
> To my understanding, it would have to be optional and off by default to
> not break other cases that are currently supported.  Think of something like
> 
>   tmp:
>       mkdir tmp
> 
>   tmp/foo.pdf: foo.tex | tmp
>       pdflatex -output-directory tmp $<
> 
> In this scenario, the tmp directory needs to be created if missing.

Actually this would not be a problem, I would need to ignore the 'tmp'
target iff 'tmp/foo.pdf' existed and was more up-to-date than 'foo.tex'.
So in cases like that the behavior even wouldn't change at all.
I think that the problem could arise in cases like this:

Dirs: dir1 dir2

dir%:
        mkdir $@

dir2/foo: dir1/bar
        touch $@

dir1/bar: | Dirs
        touch $@

.PHONY: Dirs

If you make 'dir1/bar', then remove 'dir2' and decide to make
'dir2/foo', you appreciate that 'dir2' is remade because 'dirs' is an
order-only dep of 'bar' and it is remade as soon as make realizes that
'dirs' is not complete because 'dir2' is missing.
So although this can be considered as the Makefile bug, cases like this
can exist, so the proposed change would break this behavior (I propose
that the 'Dirs' order-only dep is ignored because dir1/bar is all right
and dir2/foo says that it depends only on dir1/bar)

> > How difficult could be writing a patch that would enable this?
> 
> No idea without having a closer look.  To be honest, I am not yet
> convinced that this feature would be of real use to other users or that
> a new a global switch would be the best way to address this issue, if
> really wanted.

I think that this is a nice idea, since 'make' doesn't play well with
"filters" that can process batches of files at once, the server process
is a good way to reduce overhead of processing those files one by one
and you may end up needing a cache quite soon :-)

Actually I might be wrong, but I think that if I have a program 'filter'
capable of processing in01 in02 in03 ... files to out01 out02 out03 ...
etc., it is not possible to tell this to 'make', so if out05 and out07
are needed at some point, 'make' would call 'filter in05 in07 --some
--flags'. Is that right? Because if this was somehow possible, I would
not need that process at all.

Matej