Re: How did you handle making a GNU/Linux distribution?

[Philip McGrath]

On 9/12/21 11:09 PM, Christine Lemmer-Webber wrote:
> Philip McGrath <philip@philipmcgrath.com> writes:
> > Christine Lemmer-Webber had floated the idea at some point of trying
> > to integrate Racket and Guile.
> >
> > IIRC, I think what she's had in mind was trying to make a Guile
> > backend for Racket along the lines of the Chez Scheme backend (or the
> > BC backend, or experimental backends like Pycket).
>
> Yes that's what I had in mind :)

A few stray thoughts, all with the caveat that I haven't actually tried 
any of this ...

> > there are two things that have struck me
> > as downsides:
> >
> >   1. Flatt et al. say in "Rebuilding Racket on Chez Scheme (Experience
> >      Report)" (§6, p. 13) that, "If our task were to compile Racket to an
> >      existing target, then we would not have achieved such a high degree
> >      of compatibility. … we have taken the liberty of modifying Chez
> >      Scheme to make it an easier target for Racket."
> >
> >      https://www.cs.utah.edu/plt/publications/icfp19-fddkmstz.pdf
> >
> >      Presumably a Racket-on-Guile project would face the same trade-off,
> >      where modifications to Guild, if Racket CS is a guide, could require
> >      hard work over many years, and lesser compatibility would make the
> >      result less useful.
>
> At one point when I spoke to Matthew, he was very optimistic that the
> work on porting on top of Chez would open Racket to running on top of
> many other backends.  But yes... since there have been so many "custom"
> modifications to Chez, it's easier to be skeptical about that these
> days...

I think there a few possible senses for what "running on top of many 
other backends" could mean. My impression overall is that it has gotten 
easier, but may not necessarily be easy.

The most clearly demonstrated is that it seems to be easier to port Chez 
Scheme to new architectures than to port Racket BC's VM. Racket CS 
supports the Apple M1 chip natively (and hopefully will support the 
Linux kernel on the M1 when that stabilizes), which Racket BC does not. 
Racket CS also fully supports some platforms (ARM in general, IIRC) on 
which Racket BC lacks support for futures, places, or the JIT. The most 
promising route to Racket on WASM also seems to be adding a WASM backend 
to the Chez Scheme compiler. (In fairness, there are also some 
architectures, like ppc64le, to which no one has ported Racket CS yet, 
for which Racket BC can offer at least some support via C.)

More generally, the "linklet" abstraction[1] seems to provide a much 
more clear boundary between the responsibilities of a backend 
implementation and the common frontend than existed before Racket 7.0. 
For example, the backend doesn't have to deal with macro expansion or 
even shadowing of variables: it just need to have some way to 
instantiate linklets (likely with further backend-specific compilation) 
and to provide a whole bunch of primitives. I believe I've heard Sam 
Tobin-Hochstadt say that linklets have made it possible for Pycket (the 
experimental Racket implementation in Reticulated Python) to run most 
Racket code.

[1]: https://docs.racket-lang.org/reference/linklets.html

Another benefit of linklets is that they've defined a clear path for 
implementing parts of Racket in Racket itself, like the regexp 
implementation on Racket CS. This seems like it could help a lot with 
supplying that very large number of primitives.

So I expect it would be entirely possible to implement a linklet-based 
Racket backend in Guile. I do suspect that getting production-quality 
performance and compatibility could be a lot of work. But my bigger 
concern is ...

> >   2. As you probably know, Racket programs can't generally use
> >      Chez Scheme implemented libraries, because Chez Scheme effectively
> >      is the "unsafe" layer of the Racket VM. For example, not all Racket
> >      procedures are Chez Scheme procedures, and Racket's continuations
> >      wrap Chez Scheme's to implement delimited and composable control,
> >      threads, parameters, full continuation marks, etc.
> >
> >      For Racket CS, this isn't a great loss (there aren't so many
> >      Chez-specific libraries, and portable libraries can run in Racket's
> >      R6RS language), but, for a hypothetical Racket-on-Guile,
> >      bidirectional interoperability would be a big attraction: imagine
> >      Guix, Shepherd, Mcron, syntax/parse, racket/contract, and more all
> >      working together.

I don't think a Racket-on-Guile that worked like Racket-on-Chez would 
achieve the most interesting benefit (IMO) of bringing the two languages 
together: safe, bidirectional interoperability.

Probably there are many ways to achieve that goal. In principle, one 
could write a Racket-on-Guile implementation where arbitrary Guile could 
run without breaking Racket's invariants, but I expect that would make 
the task even harder: presumably that's why Racket-on-Chez doesn't work 
that way.

But if linklets are as good as an abstraction as they seem to be for 
compiling code in the presence of modules and macros---all of the 
complications of Scheme---then a more exciting direction, it seems to 
me, would be to adapt Guile's compiler tower[2] to, at some layer, 
produce linklets. If the existing Guile backend (which does have some 
strengths) became a linklet engine, it could run linklets generated by 
the Racket front-end. Linklets from the Guile frontend could likewise 
run on other Racket backends: Racket-and-Guile-on-Chez, Pycket, or any 
exciting backend someone creates in the future.

[2]: https://www.gnu.org/software/guile/manual/html_node/Compiler-Tower.html

It would probably still be a fair bit of work, particularly in figuring 
out exactly how the semantics of Guile modules correspond to Racket 
modules and in making interoperability convenient. I have no particular 
plans to work on this myself. But I think it would be cool :)

-Philip