[bug#70065] [PATCH 4/6] gnu: chez-scheme: Update to 10.0.0.

[Skyler Ferris]

On 3/31/24 13:17, Philip McGrath wrote:
>> The loko.scm file was not recompiled so it searched chez.scm for a
>> variable named chez-scheme-for-system, which is now a syntax
>> transformer. But compiled files have already completed the expansion
>> phase, so this means that a syntax transformer is invalid. This will not
>> cause a problem for `guix pull` because guix rebuilds itself from
>> scratch when it gets updated.
>>
>
> On 3/31/24 10:35, Skyler Ferris wrote:
>   > I haven't worked with define-deprecated before. It's a macro, and I know
>   > that the guile compiler misses recompiling macro dependencies sometimes,
>   > so maybe it was a problem on my end with having mismatching build
>   > artifacts.
>
> Very interesting! I hadn't realized that problem with dependency
> tracking existed: it might explain some issues I've struggled to
> reproduce in the past. It's also given me several surprises about
> Guile's implementation of phases: for one thing, apparently
> (define-syntax x 42) expands to (define x (make-syntax-transformer 'x
> 'macro 42)). I wouldn't have expected a macro to create a value that
> would exist at runtime.
Exists yes, but it (should not be) used/referenced at runtime. I'm going 
to type some more details in case you are interested, but an important 
disclaimer is that my understanding comes from reading documentation on 
an as-needed basis and doing some work writing macros, but not a lot. I 
haven't worked with the code that actually implements these things.

I'm not sure how other schemes implement this, but Guile has a clearly 
defined separation between "compile time" and "load time". Compilation 
is when syntax transformers are used to modify source expressions into 
the expanded form (in addition to other things, like lowering to 
intermediate representations). This is discussed in the documentation 
for eval-when (1). If I define that macro and then use it within the 
same file, the place where it is used will not contain a reference to 
the syntax transformer: it will already be expanded. If I write a new 
file tomorrow which wants to use that macro, Guile will reference the 
compiled version of the module. If the syntax transformer was not 
present in the compiled module then the new file would break. Instead, 
the syntax transformer still exists in the compiled version of the file 
so that the new module can load it at expansion time. But again, once 
the new file is compiled the reference to the macro will disappear. So 
it exists at runtime, but it is only used when compiling code. For 
compiled files, it is not used at runtime. The exception is that when 
code is evaluated at the REPL, or if auto-compilation is disabled, there 
is no practical difference between compile time and load time.

I'm not entirely sure why the compiler doesn't see the dependency here. 
The section on compilation (2) says "Guile does not yet do proper 
dependency tracking" so I assume that it just hasn't been high priority 
enough for anyone to fix it. There would be trade-offs in some cases. In 
this case it is pretty easy to see that the file needs to be recompiled 
because loko.go (the compiled version of the file) contains an explicit 
reference to the macro, because it used to be a procedure. But if it was 
a macro both before and after the change the reference would not be 
there. We could add the list of macros used during compilation as 
metadata to the file but I don't know how much bloat that would cause. 
We could also reference the use-module forms in the original source, but 
I'm not sure if those are typically available in the go files or if that 
would also need to be tracked as additional metadata.

(1) https://www.gnu.org/software/guile/manual/html_node/Eval-When.html
(2) https://www.gnu.org/software/guile/manual/html_node/Compilation.html