Re: [MIT-Scheme-devel] Faster ports.

[Matt Birkholz]

> From: Taylor R Campbell <address@hidden>
> Date: Fri, 24 Jun 2011 03:10:22 +0000
> 
>    Date: Thu, 23 Jun 2011 16:06:59 -0700
>    From: Matt Birkholz <address@hidden>
> 
>    [...]
>        (lsh (r 42) half-word-size)
>        (and (r 43) type-mask)
>        (and (r 42) (r 43))                ; r42 <- address of bytes
>         ^^^
> ADD, surely?

After shifting and masking them, I think of them as unsigned bits.  Do
you mean "Use the ALU if you've already got the logic/shifter circuits
engaged."?

> [...]  I would suggest using not aliens but slices for the
> purpose. [...]  It should have the following fields, for a native
> implementation:
> 
> - frob
> - length
> - high address
> - low address
> 
> [...]
> 1. The garbage collector needs to be able to relocate the address
> fields, in case the frob is a vector-8b in the Scheme heap that moves
> around.

So slice-ref will use the two address fields regardless of buffer
type?  That makes it as slow as external-string-ref EVERY time.  Our
"near" type tags (should) make the cost of STRING? negligible so,
again, if this

        (if (string? buffer)
            (string-ref buffer start)
            (external-string-ref buffer start))

is inlined via SOURCE-OCTET, internal string buffers are negligibly
more costly, yet external string buffers are supported.  Their
unavoidable* expense is incurred only when necessary.

> 2. The garbage collector needs to know when it can release the frob if
> it is an external string or similar, so there has to be a reference
> for the sake of a GC finalizer in that case.

I believe weak pairs is all you need.  I implemented this property for
malloced aliens without special garbage collector support.

> On 64-bit systems, it would be nice if the high address and low
> address fields were coalesced into a single field, since all addresses
> will fit into fixnums.  But that would require adding a lot of highly
> error-prone code which isn't worth experimenting with yet.

* The unavoidable expense versus coalescing: Can we really depend on a
64bit malloc not to return a bizarre address like #x0400000000000000?
Working with half-words guarantees we can represent the whole address
space (CRUCIAL on a 32bit machine).

> I didn't intend to involve ports at all.  [...]

I thought we were talking about genericizing input port ops, via an
octet-source whose buffer could be internal or external to the heap.
Such a thing would presumably replace the input-buffer in a
generic-i/o port's gstate, and all character and line-ending decoding
would source octets with SOURCE-OCTET.  Thus my GInputStreams can
enjoy character decoding, and my async buffer can stand on the
shoulders of SOURCE-OCTET's buffer management, and all I need to
provide is a FILL!? method.

> Effectively, cpress.scm already uses octet sources.  They're just
> hand-written for the particular application.  I want to capture that
> pattern into an abstraction.

Me too, with a cheap yet generic-ish operation so I can compress an
HTML::Tree right into an external (async) buffer.

> However, there are a lot of problems to be worked out.  [...]
> 
> [...]  It may turn out in practice that everything I've been
> rambling about is a net lose over using ports as currently designed.

Ha, ha.  You had me going there.  Oh, well.

I just wanted async i/o without mangling the port performance.  I
suppose I should have done

        Step 1 (first!): write tests/runtime/port-performance.scm.

        Step 2: implement the obvious extra copy from async buffer to
        internal string buffer.  Take a baseline GIO performance
        measurement.

        Step 3: genericize genio.scm so that works on external string
        buffers.  Take another GIO performance measurement, compare
        and complain. :-)

Any help with Step 1 out there?