Re: [MIT-Scheme-devel] Module system?

[Taylor R Campbell]

   Date: Sun, 30 Aug 2009 09:34:08 -0700
   From: address@hidden (Matt Birkholz)

   > From: Taylor R Campbell <address@hidden>
   > Date: Sat, 29 Aug 2009 23:01:10 -0400
   > 
   > [...]
   > The main problem here is that the compiler/linker and the macro system
   > fail to communicate, and that the macro system uses the wrong data
   > structure, interpreter environments.

   I am getting no traction from that.  Are you re-stating my problem to
   make the solution easier to visualize?  There IS a communication
   problem: fasload needs more information from the macro system in order
   to re-establish the proper links... ?

   But I do not see how using separate data structures for variables and
   keywords will help.  The interpreter environments model the lexical
   contours of the code, which are relevant to both macro expansion and
   variable reference.  ?

Interpreter environments map names to values, and macros are grafted
into this system by macro reference traps.  The compiler, linker, and
macro system should all be using environments that map names to
denotations.  The denotation of a name is, for example, a macro and
its associated transformer procedure, or a variable and (a model of)
the associated location where its value is stored.

The interpreter's data structure, for that matter, needn't be anything
beyond a list of vectors; no names need be involved, and the locations
of interpreter variables can be up/over (n contours up, m variables
over in that contour) indices.  (Changing the interpreter to do this
would probably speed it up significantly, although I imagine more work
is likely to happen sooner in the SVM.)

Here I am using the terms `compiler' and `linker' loosely, by the way.
`Compiler' includes anything that analyzes Scheme programs, such as SF
or LIAR; `linker' includes anything that prepares Scheme programs for
combination, or combines Scheme programs, such as CREF, the compiler's
linkification phase, and the microcode's compiled-code linker.

   > [...]  Without a naming authority [...] universal names are not
   > practical.

   I would stick with the naming authority we have now.

What authority is that?  The output of running `find . -name \*.pkg'
in MIT Scheme's Git repository?

   > [...]  In MIT Scheme the situation is complicated by the existence
   > of a separate interpreter which also uses the macro expander.

   A "separate" interpreter that "also uses" is a mighty fine hair.
   Again, I am not sure where you are going with this distinction.

This is not an important semantic distinction -- it's just that
Scheme48's macro expander is tied fairly tightly to the compiler, so
that it has more leeway in the format of its output, because nothing
but its only client, the compiler, expects to deal with it.

   > [...]  CREF is not really an appropriate starting point -- aside
   > from its failure to record actual dependencies, rather than just
   > environment linkage, in order for any of this to work well, the
   > compiler/linker and macro system have to communicate.

   You must have a lot of big problems that you want the module system to
   solve.  What are these "actual" dependencies?  Build dependencies?

;;; foo.scm

(define foo (+ bar 1))

;;; bar.scm

(define bar 5)

;;; test.pkg

(global-definitions "/path/to/runtime/runtime")

(define-package (test)
  (parent ()))

(define-package (test foo)
  (parent (test))
  (files "foo")
  (export (test) foo))

(define-package (test bar)
  (parent (test))
  (files "bar")
  (export (test) bar))

;;; REPL

(sf '("foo" "bar"))
;Generating SCode for file: "foo.scm" => "foo.bin"... done
;Generating SCode for file: "bar.scm" => "bar.bin"... done
;Unspecified return value

(cref/generate-constructors "test")
;Dumping "test-unx.pkd"... done
;Unspecified return value

(load-package-set "test")
;Loading "test-unx.pkd"... done
;Loading "foo.bin"... 
;Unassigned variable: bar

What went wrong?  Nowhere in the CREF language can I express the
dependency that foo.scm must be loaded before bar.scm.  (Since CREF
allows mutually referential packages, it's not enough to observe that
(TEST FOO) uses bindings from (TEST BAR).  That is the kind of
dependency that one expresses in, for example, Scheme48 and PLT
Scheme.)

   I don't expect CREF to model build dependencies because I know they
   include all the little .cdecl files my C-include macro loaded.  My
   build process has to ensure that all files mentioning (C-sizeof
   "GdkColor") get re-compiled when the C-sizeof macro definition changes
   OR the C declaration in gdkcolor.cdecl changes.

Why shouldn't a module system allow you to express this information,
and use it?

   I don't want to solve any BIG problems.  I just want to be able to
   fasload a reference to a variable in a named environment.

   I guess my work will NOT relate to any serious module system effort. ;-)

Whenever I hear about non-serious efforts I become afraid that they
will grow and accrue and thereby inhibit serious module system
efforts.