Re: Update on the ‘wip-gash’ branch

[Timothy Sample]

Hi Janneke,

Janneke Nieuwenhuizen <janneke@gnu.org> writes:

> I have added a couple of squash! commits to ressurrect M2-Planet build,
> the debugging simple.make and simple.sh builds, and made some minor
> changes (prefix work in src/ with "core:" instead of "mes:", add
> some Copyright lines, mention you in AUTHORS an from the repl, etc.).
>
> Planning now is to release this as Mes 0.26, we'll have quite some time
> to change or add things.

Thanks for taking a look.  It feels great to have a plan in place and be
“officially” on the release schedule!  For my part, it would be really
nifty if 0.26 means that both Gash and Gash-Utils are Mes-powered in
Guix’s bootstrap.  That means I will be working away at getting
Gash-Utils running on Mes.

> Timothy Sample writes:

>>   • When Autoconf tries to find out how to print, it does a test that
>>   involves many hundreds of backslashes.  While trying to process all
>>   these escapes using the usual list processing procedures (‘map’ and
>>   friends) the Mes Scheme stack would overflow.  To fix this, I rewrote
>>   many list processing procedures to be iterative instead of recursive.
>
> Yeah, I've been wondering how that works.  "loop" is implemented as a
> lamdda that also recurses...
>
> (define-macro (simple-let bindings . rest)
>   (cons (cons 'lambda (cons (map1 car bindings) rest))
>         (map1 cadr bindings)))
>
> (define-macro (xnamed-let name bindings rest)
>   `(simple-let ((,name *unspecified*))
>      (set! ,name (lambda ,(map1 car bindings) ,@rest))
>      (,name ,@(map1 cadr bindings))))
>
> ...so while it's great that you found this solution, I'm a bit puzzled.
> Of course, Mes should get tail call elimination support some time ;-)
> Until then, I guess these changes are fine.

Mes does not add stack frames when invoking a procedure in tail
position.  I guess you implemented “tail call elimination” by accident!
:)  (If you’ll indulge me in a little rant, this doesn’t surprise me.
Scheme is designed so that “tail call elimination” is the natural thing.
That’s why I keep putting it in scare quotes.  In my mind Scheme doesn’t
do “tail call elimination”.  It’s other languages that do “tail call
introduction”!)

Here’s a little demo.

(define (max-stack-depth-iter n acc)
  (if (zero? n)
      (apply max acc)
      (let ((depth (stack-length (make-stack))))
        (max-stack-depth-iter (- n 1) (cons depth acc)))))

(define (max-stack-depth-rec n)
  (if (zero? n)
      0
      (let* ((depth (stack-length (make-stack))))
        (max depth (max-stack-depth-rec (- n 1))))))

(display "max-stack-depth-iter (100): ")
(display (max-stack-depth-iter 100 '()))
(newline)

(display "max-stack-depth-rec (100): ")
(display (max-stack-depth-rec 100))
(newline)

When I run this script on Mes, I get:

    max-stack-depth-iter (100): 11
    max-stack-depth-rec (100): 308

If I adapt it for Guile (by passing “#t” to “make-stack”), I get:

    max-stack-depth-iter (100): 9
    max-stack-depth-rec (100): 108

Mes has a few bugs here – I believe using “apply” in tail position could
overflow the C stack, for instance – but the fundamentals are sound.

> Ah right.  Hmm.  Well, let's see if that's enough.  How difficult (or
> costly) would it be for the garbage collector to go through the dead
> cells to close ports?

I noticed a little while ago that Zuo [1] has a distinct “handle” type.
Starting with something like that, the garbage collector could tell the
difference between numbers and file handles.  It doesn’t have to be a
basic type: it could be a record.  We could keep a special table of
handles.  Periodically, we could run GC (with the collector marking all
reachable handles), and then scan the table to close any unreachable
handles.

That’s off the top of my head.  I’m sure doing a little research about
the interplay of stop-and-copy GC and finalizers would give us something
to work with.

[1] https://github.com/racket/zuo

> I have also added
>
>     519a690b * squash! DRAFT core: Add a C-only module lookup fast path.
>     4e7d290c * squash! mes: guile: Remove 'read-line'.
>
> that remove code that you commented-out.  Any reason for keeping that
> commented code, otherwise I'll squash them too.

No reason.  Delete away!

>> That includes just the module system work.  I tested that the other day,
>> and was able to build everything “--with-bootstrap” using “boot-5.scm”
>> as the default.  I had to mark “psyntax.test” as an expected failure,
>> but all the other tests (the full “make check”) pass.  To be clear, the
>> ‘wip-gash’ branch also passes all tests, but I haven’t tried using
>> “boot-5.scm” (I see no reason why it wouldn’t work).
>
> That's awesome!  Looking at wip-modular-mes, please feel free to move
> any/more compatibility, support or stubs to Mes if you like (usleep,
> fold-right, string-for-each, ...).

Will do.  I will probably need a lot of it for Gash-Utils.

> I'm wondering now that boot-5.scm works *), should we remove the unused and
> flaky psyntax support and default to boot-5.  Possibly we could even
> remove the mes-use-module "feature" altogether,  WDYT?

Having them both was good while developing, since I could check
performance and compatibility against the old version.  When we fully
commit to the full module system, I see no need to keep the old version
around.

> *) The only thing that I found not to work is using (use-modules ...)
>    from the repl, so we'd need some fix or workaround for that possibly
>    a new ,use command?

The old Guile module system only allows running “use-modules” at the top
level.  Because of how the REPL is implemented, it is not considered
“top level” (even though it should be).  I had to rewrite the
“eval-case” macro that the module system uses to determine if code is at
the top level.  It works, but maybe we need to coordinate with the REPL
so that it can be spoofed or something.


-- Tim