[Chicken-hackers] [PATCH] Fix meta evaluation (so that require-extension-for-syntax works properly)

[Peter Bex]

Hi all,

There's still a problem with the new require-extension-for-syntax form,
as Felix pointed out to me:

#;1> (require-extension-for-syntax tabexpand)  
#;2> (define-syntax foo (lambda (x r c) (print (tabexpand "abc")) 1))         
Note: the following toplevel variables are referenced but unbound:              
                            

   tabexpand

I've dug into this, and it turns out that the problem is really in the
way begin-for-syntax interacts with "import":

#;1> (require-library tabexpand)    
#;2> (begin-for-syntax (import tabexpand))
#;3> (define-syntax x (lambda _ (tabexpand "foo")))
Note: the following toplevel variables are referenced but unbound:

  tabexpand

I'd expect (begin-for-syntax (import foo)) to be equivalent to
(import-for-syntax foo).  The way begin-for-syntax is implemented
is that it expands to (##core#elaborationtimeonly (##core#begin ...)).

This core form is implemented via eval/meta, in eval.scm:810.  That
raises the evalation meta-level by one (conceptually speaking; in
practice it only works for one level), and then evals the given form.

On the other hand, import expands to a call to ##sys#expand-import
with the environments ##sys#current-environment and ##sys#macro-environment,
whereas import-for-syntax expands to an almost(!) identical call,
but with ##sys#current-meta-environment and ##sys#meta-macro-environment.
You can see this in expand.scm:911

Finally, ##sys#expand-import just loads up the library and puts the
loaded bindings into the given environments.

The problem is that eval/meta did not override the current environment
with the meta environment.  This meant that that effectively, imports
were *always* done into the normal environment.  However, for some
unknown reason, lookups happen in ##sys#active-eval-environment rather
than in ##sys#current-environment (which gets raised correctly to the
current-meta-environment).  So the import happens into the "ground level"
environment, while the lookup (in ##sys#alias-global-hook, in
modules.scm:747) happens into the meta-environment, which causes a
mismatch.  You can see the import into the normal environment by looking
up the identifier:

#;1> (require-library tabexpand)
#;2> (begin-for-syntax (import tabexpand))
#;3> tabexpand  ; should cause "unbound variable: tabexpand"
#<procedure (tabexpand#tabexpand str43)>

This is wrong; this identifier is supposed to get imported into the
meta-environment but instead it got imported into the toplevel
environment.

All this had a result of making require-extension-for-syntax fail as
well.  I've attached a fix which simply raises current-environment
along with active-eval-environment.  Upon restore, it writes the
current environment into the meta environment, so that any identifiers
that were imported into the normal environment during the eval/meta
get stored back into the meta environment.

In the long run, I'd like to look into reducing the number of parameters.
For example, it looks like ##sys#active-eval-environment gets used *only*
in the ##sys#alias-global-hook, and nowhere else.  I think that's
probably unneccessary, as it can use ##sys#current-environment equally
well.  The fact that ##sys#active-eval-environment is a parameter
*holding another parameter* which itself holds an environment just
causes my head to spin even more.

I'm also unsure as to why there's a distinction between
##sys#current-meta-environment and ##sys#current-macro-environment.
Oh, and then there's ##sys#chicken-macro-environment and
##sys#chicken-ffi-macro-environment and of course the normal syntax
environment associated with identifiers.  Removing some of these
may also help us get rid of some of the more esoteric bugs that have
been reported, and possibly gain us proper support for multiple
evaluation levels per module (currently all the macro environments
are shared between all modules and there's only one level, which
means you can't import a module for syntax which itself imports
another module for syntax and have it see a clean environment).

Cheers,
Peter
-- 
http://www.more-magic.net

0001-Fix-meta-evaluation-to-actually-take-place-in-the-me.patch
Description: Text document