[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]
bug#27284: ‘guix pull’ builds using multiple derivations
From: |
Nils Gillmann |
Subject: |
bug#27284: ‘guix pull’ builds using multiple derivations |
Date: |
Tue, 27 Mar 2018 19:25:18 +0000 |
Awesome news!
Ludovic Courtès transcribed 3.1K bytes:
> Hello Guix!
>
> News from the front! I’ve pushed a new ‘wip-pull-multiple-derivations’
> branch that keeps the same approach as previously proposed (building
> Guix using multiple derivation, one for each group of modules: core,
> CLI, packages, etc.), but does not attempt to reload modules in the
> running Guile process, which proved to be too tricky.
>
> You can already test with:
>
> guix pull --branch=wip-pull-multiple-derivations
>
> Feedback welcome!
I think you forgot to push it or you are on a very slow connection :)
> This patch set actually addresses a bootstrapping issue: you need Guix
> to build Guix. So far ‘guix pull’ would sidestep the bootstrapping
> issue by building Guix with whatever is available in the currently
> deployed Guix; for instance, it would use the ‘guile’, ‘guile-json’,
> etc. packages from the current Guix. The problem of that approach was
> that it’s stateful: the result depends on what you currently have.
> Sometimes what you have is too old, or lacks some package definition,
> and you can’t really go forward.
>
> The code in ‘wip-pull-multiple-derivations’ works like this:
>
> 1. Assume we have Guile and Guix already installed, but not
> necessarily the latest versions thereof.
>
> 2. Using the (guix …) modules that we have, build a program—a
> “trampoline”—that will use the modules of the target Guix (the
> commit we want to pull) to compute the derivation of that Guix.
>
> 3. Run that trampoline, which returns /gnu/store/…-guix.drv. The
> result should be the same regardless of the initial Guix because
> the trampoline uses exclusively modules from the target Guix.
>
> 4. ‘guix pull’ builds that derivation (actually the branch does not
> modify (guix scripts pull) at all; everything is in
> build-aux/build-self.scm.)
>
> In step #4, we should be able to get substitutes for at least some of
> the derivations.
>
> To build the trampoline in step #2, we first need to build a bunch of
> modules from the target Guix. Hopefully you don’t have to rebuild them
> at each pull, but it can take a minute or so, and you may not have
> substitutes for that (because this part is stateful.)
>
> In step #3, the trampoline has compiled code for the core modules, but
> it still has to interpret (gnu packages guile) and related modules,
> because these are not compiled. The program runs in ~40s on my laptop.
> To make it faster, we could reduce the closure of (gnu packages guile)¹,
> but that can be tricky. Or we could optimize Guile itself; I’m sure the
> compiler and/or interpreter could do better.
>
> In that branch, you can also run:
>
> make as-derivation
>
> and it will build Guix from $srcdir in the way described above.
>
> To summarize, performance is not great, but hopefully it’ll be slightly
> better than what we have now, especially with substitutes.
> Correctness/reproducibility are greatly improved.
>
> Ludo’.
>
> ¹ The closure of (gnu packages guile) contains python, haskell-check,
> music, and many other surprising things:
> <http://web.fdn.fr/~lcourtes/tmp/gnu-packages-guile.html>.
>
>
>