[bug#38390] [core-updates] Scheme-only bootstrap: merge wip-bootstrap

[Ludovic Courtès]

Howdy!

Jan Nieuwenhuizen <address@hidden> skribis:

>> We will be working on a rewrite of wip-bootstrap to have it use Gash
>> wip-0.2.0+ and include a number of cleanups.
>
> I have just hard-reset wip-bootstrap for the next iteration!

Yay!

>> Here is my current TODO list
>>
>>   * base bootstrap on Gash wip-0.20.0 (plus janneke's 2 patches)
>
> Done.  I just built `hello', so Gash 0.2.0 here we come (afaic).
>
>>   * remove %bootstrap-gash (with gash core utils) from bootstrap seed
>
> Done; thanks to Ludo's insights about %boostrap-guile+guild, we were
> able to do this.  The `gash-boot' and `gash-core-utils-boot' packages no
> longer appear in shells.scm; they are built during bootstrap using the
> guile-build-system.  I currently add lalr.upstream as an extra origin
> to gash core utils from
>
> "http://git.savannah.gnu.org/cgit/guile.git/plain/module/system/base/lalr.upstream.scm?h=v2.0.9";

Awesome.

BTW, I’d suggest this for clarity:

diff --git a/gnu/packages/commencement.scm b/gnu/packages/commencement.scm
index 9e3cecc174..d4cf58a7eb 100644
--- a/gnu/packages/commencement.scm
+++ b/gnu/packages/commencement.scm
@@ -266,6 +266,7 @@
                       ,(origin
                          (method url-fetch)
                          (uri (string-append 
"http://git.savannah.gnu.org/cgit/guile.git/plain/module/system/base/lalr.upstream.scm?h=v2.0.9";))
+                         (file-name "lalr.upstream.scm")
                          (sha256
                           (base32
                            
"0h7gyjj8nr2qrgzwma146s7l22scp8bbcqzdy9wqf12bgyhbw7d5"))))))

>>   * look at possibility/cost to avoid updating the mescc-tools and mes
>>     bootstrap binaries
>
> This proved possible (dare I say feasible?).  What was needed is
>
>  - allow mescc (0.21+) to work with old mescc-tools 0.5.2
>  - add %bootstrap-mes-rewired, to remove MesCC from mes-0.19 and
>    enable it to run 0.21+ MesCC
>
> %bootstrap-mes-rewired uses this hack
>
>          (add-after 'unpack 'patch-%moduledir
>            (lambda _
>              (copy-file "module/mescc.scm" "module/mescc.scm.orig")
>              (substitute* "module/mescc.scm"
>                (("^  \\(mes-use-module \\(mescc mescc\\)" all)
>                 (string-append "
>   ;; MesCC from mes-0.21
>   (let* ((self (car (command-line)))
>          (prefix (dirname (dirname self))))
>     (set! %moduledir (string-append prefix \"/mes/module/\"))
>     (setenv \"%numbered_arch\" \"true\"))
>
>   ;; A fixed map, from mes-0.21
>   (define (map f h . t)
>     (if (or (null? h)
>             (and (pair? t) (null? (car t)))
>             (and (pair? t) (pair? (cdr t)) (null? (cadr t)))) '()
>         (if (null? t) (cons (f (car h)) (map f (cdr h)))
>             (if (null? (cdr t))
>                 (cons (f (car h) (caar t)) (map f (cdr h) (cdar t)))
>                 (if (null? (cddr t))
>                     (cons (f (car h) (caar t) (caadr t)) (map f (cdr h) (cdar 
> t) (cdadr t)))
>                     (if (null? (cdddr t))
>                         (cons (f (car h) (caar t) (caadr t) (car (caddr t))) 
> (map f (cdr h) (cdar t) (cdadr t) (cdr (caddr t))))
>                         (error 'unsupported (cons* 'map-5: f h t))) )))))
> " all)))
>              #t))
>
> ...not particularly nice/clean/auditable; but "it works".  WDYT?

‘caddr’, loooove it!  ;-)

It seems to me that mescc.scm in 0.21 only use the two-argument ‘map’,
no?  In that case I guess we could go with a two-argument, fixed-arity
version that would have fewer cdadrs caddrs and caadrs?  Just sayin’.
:-)

> On a related note, should mescc 0.21 include some kind of `hook' make
> this kind of change easier to make?

Good question!  During the summit, Vagrant (I think?) asked whether Mes
and MesCC should be separated.  I think in this case it would have been
beneficial to have them distributed separately, no?  It might be worth
looking at which parts change frequently to determine what to keep as
part of the ‘mes’ package itself.

Thoughts?

>>   * remove any generated (gitlab/github) tarballs
>
> Done (but please check!).

LGTM!

>>   * look into awkward combined bash+gash dependency of glibc-mesboot0
>
> Haven't addressed this.  I quickly looked with Ludo at this, not really
> into it though.  WYDT?

Hmm, dunno.  I can take a look later.

>>   * add some %bootX-input stages, at least when reached gcc-mesboot1
>
> Done.  Numbering of bootX, mesbootX could possibly made to make more
> sense.  However, I also would like to get rid of the whole 2.95 story
> some time soon and then many things change again.  I could do with some
> help/inspiration here.

I think it’s good to have ‘%bootX-inputs’ for when we’re going to reuse
more or less the same input set several times, because it clarifies that
there’s a “plateau” of sorts, but I guess it’s not always appropriate
early on in the graph.

Another thing we discussed was the growth of the dependency graph:

--8<---------------cut here---------------start------------->8---
$ ./pre-inst-env guix graph -e '(@@ (gnu packages commencement) 
coreutils-final)' | grep 'label =' | wc -l
177
$ guix graph -e '(@@ (gnu packages commencement) coreutils-final)' | grep 
'label =' | wc -l
76
--8<---------------cut here---------------end--------------->8---

Clearly we’re adding way more nodes than we mean to.

For example, you can remove the call to ‘package-with-bootstrap-guile’
for ‘bash-mesboot’ (down to 153 nodes) and the one for
‘gcc-core-mesboot1’ (down to 121) and the one for ‘binutils-mesboot’
(down to 87), and I think at this point we’re done and the graph has a
lovely shape.  :-)

It makes a noticeable difference from a performance viewpoint (see
<https://lists.gnu.org/archive/html/guix-devel/2019-10/msg00350.html>.)

Also, you can leave out calls to ‘bootstrap-origin’ for origins without
a snippet and without patches (it’s purely stylistic though.)

> We are really getting there!  I would like to do a rewrite once Gash
> 0.2.0 is out.  What to do about Gash Core Utils?  I also plan to do a
> rewrite once MES 0.22 is out.

I think we should probably wait for the Gash and Gash Core Utils
releases so we can refer to them directly.  How does that sound to you,
Timothy?

Thanks!

Ludo’.