[MIT-Scheme-devel] retractable I/O thread event registration, and rendezvous

[Taylor R Campbell]

I just hacked up a way to generate CML rendezvous, using my library at
<http://mumble.net/~campbell/darcs/scm-rendezvous/>, for input and
output.  Unfortunately, it uses some rather grody internal hacks, and
I think I may have uncovered a minute bug in the I/O thread event
code.

Here's the crux of the code:

   (define (channel-io-rendezvous channel mode)
     (polling-rendezvous
      (lambda ()
        (cond ((test-for-io-on-channel channel mode #f)
               => (lambda (mode)
                    (values 1
                            (lambda (prepare-revival)
                              prepare-revival ;ignore
                              mode))))
              (else
               (values #f
                       (lambda (suspension)
                         (let* ((token (suspension-token suspension))
                                (thread (car token)))
                           (register-io-thread-event
                            (channel-descriptor-for-select channel)
                            mode
                            thread
                            (lambda (mode)
                              (%revive thread suspension token
                                       (list mode))))))))))))

This seems to work for my very, very simple tests, but it has three
problems:

1. It totally violates the suspension abstraction, but this is only
   because the I/O thread events need to be delivered to *some*
   thread, and the thread being suspended is the only one I can think
   of that makes sense.  What would be nice would be to integrate the
   suspension abstraction into the system.  I'll mull over the
   ramifications of this, but I think that it would be an overall win.
   (Suspensions are more general than condition variables.)

2. I don't quite understand the select registry well enough to say for
   certain, but I suspect that it doesn't clean up properly after
   subprocess status changes.  Since HANDLE-SUBPROCESS-STATUS is not
   exposed, though, this is hard to do correctly.  (Incidentally, why
   doesn't TEST-FOR-IO-ON-CHANNEL or TEST-FOR-IO-ON-DESCRIPTOR just
   handle it internally?)

3. It doesn't cancel stale I/O registrations, so even if the channel
   rendezvous is discarded and another rendezvous chosen instead, the
   I/O registration will remain in there, and the descriptor will
   remain in the select registry, wasting resources.

Do the latter two problems sound accurate?  I only recently started
investigating how the internal select registry works, so my analysis
of it could easily be bogus.  Also, any thoughts about moving the
suspension abstraction into the run-time system so that it can be used
ubiquitously to build higher-level concurrency devices conveniently.
(I can explain what it is precisely if necessary.)

By the way, I'd be interested to hear whether anyone has tried my
rendezvous system out, or would like to try it out were there
documentation.  Despite the design of MIT Scheme's existing thread
system (which, unfortunately, I probably sha'n't get around to
finishing by the end of this summer), the rendezvous system works
perfectly well at least for small examples, and I'd like to see how it
scales as a decent, high-level concurrency abstraction for larger
applications.

(Unfortunately, the rendezvous system can't be compiled, due to a
problem with the fasdumper and macro-generating macros.  I couldn't
find any workaround (aside from rewriting the macros and all their
usage sites, which is not really an option), so for the time being it
will have to be run interpreted.  I'll investigate it again at some
point, though.)