[Chicken-hackers] [PATCH] Don't evaluate import forms for syntax in import libraries when there is no syntax exported (mostly fixes #1457)

[Peter Bex]

Hi all,

After several stabs at investigating the issue in #1457, I finally
figured out the root cause (besides the obvious quadratic behaviour
in merge-se): 

If a module imports foo, we emit (eval '(import-syntax foo)) in the
import library for that module.  The idea here is that this might
be necessary if the module exports syntax forms, because a macro
transformer in principle runs arbitrary code that expands to
arbitrary identifiers.

The macro expander may need any of the things that are imported
into the module where it's defined (it can call arbitrary procedures).
That's why we do (import ..) for every module that is imported for syntax.

Because the macro expander may also expand to arbitrary procedure calls,
we need to ensure that the bindings when expanding such a macro also
correspond to the bindings imported inside the module where it's defined.
That's why we do (import-syntax ..) for every module that is imported
in the module where syntax is defined.

All of this is completely unnecessary if the module exports no macro
at all!  The attached patch simply emits the import forms only if there
actually is syntax exported from the module for which we're generating
the import library.

This doesn't fix the actual O(n^2) behaviour of merge-se, but if we can
avoid calling it as much as possible, this should at least make it
managable.


Note that currently in register-compiled-module, we bluntly attach the
entire module environment to each macro that the module exports, which
just results in huge syntax envs in memory and requires us to import all
the modules imported by the module being registered.  We could also
_emit_ the pre-mapped syntax envs into the import libs to avoid all
the imports, but this would result in very large import files.

Going forward, I think a nicer approach could be to explicitly store
mappings of the syntax with each macro like we do in chicken-syntax.scm.
This would avoid the need for imports for the identifiers in a
macro's _output_.

This would have to be an opt-in feature, as we don't want to burden the
user with making these (error-prone!) mappings, but at least it would
provide a nice "out" of terrible run- and compilation time speeds when
using lots of modules.  We could also look into extending syntax-rules
to *automatically* do this (I think that's quite doable!).  We also know
that syntax-rules don't run arbitrary code at expansion time (it only
uses several well-known identifiers).

PS: If you want to test the performance boost of this properly, you must
first recompile CHICKEN with itself so that any of our generated import
libs won't contain evals either, and if you want to test with eggs you
need to remove all eggs and ensure the chicken-install cache dir is empty
as well as your local checkout of the egg repo, otherwise you'll
inadvertently be using old import libraries which still contain evaluated
import expressions!

Cheers,
Peter

0001-Do-not-emit-import-syntax-forms-when-no-syntax-forms.patch
Description: Text Data

signature.asc
Description: PGP signature