Re: [MIT-Scheme-devel] Faster ports.

[Matt Birkholz]

> From: Taylor R Campbell <address@hidden>
> Date: Thu, 23 Jun 2011 02:41:11 +0000
> 
> [...] I don't expect the overhead of branching and EXTERNAL-STRING-
> REFing to be negligible even if open-coded.

I suppose there is an unavoidable extra indirection in the
external-string case.  I thought YOU wanted both internally and
externally buffered ports.  We inline a buffer type test,

        (if (string? buffer)
            (string-ref buffer start)
            (external-string-ref buffer start))

or go all external?

        (external-string-ref buffer start)

>    How about calling them "bytes"?  It is shorter than "octets", yes?
> 
> `Byte' won't make any sense when we get the PDP-10 port working again.

That very old joke is why you can't say "byte"?  You can't be serious.

> How about `u8' as a compromise?  It extends nicely to `s8', `u16le',
> &c.

I would hate to see it get any shorter.

>    If buffer transfers are all inlined via integrable calls to READ-OCTET
>    etc., will our ports smoke?  Did you have something else in mind?
> 
> Well, there are a lot of problems with ports and I/O in general in MIT
> Scheme...

You say "problems" and I say "puzzles".  Let's call the whole thing off! :-}

> - Octet-by-octet reads and writes with ports are out-of-line [...]

We're talking about fixing that, yes?

> - The port concept isn't a very clear abstraction: [...]

It's a bunch of them, yes.  I assume high-performance code will avoid
the most generic procedures like read-char.

> - There is no support for memory-mapped I/O [...]

Whoa.  Don't get the portability drums started...

> - The whole thread I/O event system is a kludge [...]

A kludge that works is a win.  It has not held me back.

> At <http://mumble.net/~campbell/tmp/mit-io.tgz> is a tarball [...]

You are keeping buffers in sources/sinks?  I thought you wanted them
"exposed" to the inlined read-u8.

> I compared ports [...] the improvement in performance [...] was
> enormous -- probably a decimal order of magnitude, at least.

How did/would you take measurements?

> Here's a very rough sketch of the plan I have had in mind [...]
> 
> 1. Expose something similar to the channel abstraction, [...]

I thought we had "exposed"(?) the buffer (not the "channel"), to the
inlined code, so that e.g.

        (write-u8 output (read-u8 input))

could be totally integrated.  If the input/output's buffer is not
empty/full, a little fixnum arithmetic happens and then:

        (external-string-u8-set!
         target-buffer target-start
         (external-string-u8-ref source-buffer source-start))

OR the more generic

        (let ((fixnum (if (string? source-buffer)
                          (string-u8-ref source-buffer source-start)
                          (external-string-u8-ref source-buffer source-start))))
          (if (string? target-buffer)
              (string-u8-set! target-buffer target-start fixnum)
              (external-string-u8-set! target-buffer target-start fixnum)))

> 5. Change Edwin to have a more understandable event loop, [...]

I hear that.  I made a few changes to accommodate <gtk-screen> and
avoid spin-waits...

> But enough rambling.  I've been trying to adhere to a discipline of
> less talk, more code, and I haven't worked up the courage to dive
> into all these problems, so I've been keeping pretty quiet about
> them...

I'm doing async i/o via ports, so I hacked external string buffers in,
but I'd like a better implementation, for my Gtk branch (if not
master).