[Visionaries] Local RPC

[Peter Minten]

Hi folks,

I've been thinking about a way to connect WebScheme and C# code. A fast approach
would be direct C type calls. Disadvantage is that a lot of type checking and
conversion has to be done. A flexible approach would be XML-RPC, but that's even
slower.

I believe a mix form can be used for languages on one box to communicate. The
basic idea is that XML-RPC is pretty good, but the XML needs to be replaced by a
binary protocol (can be parsed faster). Another idea is that for every object
that cannot be easily translated tickets are used.

Tickets are opaque shadow objects. A ticket uses advanced reflection techniques
to transform a method call like ticket.foo to an RPC call (ticket_nr, "foo"). If
the RPC is done using a binary protocol it's extremely fast so tickets are fast
enough. 

Tickets are not understandable values to the client language, but the easily
translatable data types are. Given that C# is the most important language in
DotGNU I propose to create the following basic (non-ticket) data types and
binary encodings:
1 SByte
    Encoded as a single byte
2 Byte
    Encoded as a single byte
3 Binary (Byte[] in C#)
    Encoded as a series of bytes in the following setup
    "length\0contents"
4 Char (2 byte, unicode)
    Encoded as two bytes
5 String
    Encoded as a multitude of two bytes in the following setup
    "length\0contents", total length of complete string part in bytes
6 Int16
    Encoded as 2 bytes
7 Int32
    Encoded as 4 bytes
8 Int64
    Encoded as 8 bytes
9 UInt16 
    Encoded as 2 bytes
10 UInt32
    Encoded as 4 bytes
11 UInt64
    Encoded as 8 bytes
12 Single
    Encoded as 4 bytes
13 Double
    Encoded as 8 bytes
14 Ticket
    Encoded as 4 bytes (it's a UInt32)
15 Array
    Encoded as a sub binary string
16 Hash
    Encoded as a sub binary string with the first of every two elements a string

The encoding strings are made up like this:
"stringlength'\0'encodingbytes'\0'contents"
where stringlength is the total length of the RPC string
where encodingbytes is a string of bytes telling the data types that are at a
certain position in the string
where contents is the contents

Take for example the following string (using escapes for byte values):
"\31\0\6\2\5\15\0\0xFE\0xD3\25\11\0\72\0\101\0\108\0\108\0\111\10\0\2\14\0\42\0x12\0x34\0x56\0x78"

Here's the same string annotated:
\31 -- Length of RPC string
\0  -- Division character because string length can be really big
\6  -- First content chunk is a Int16
\2  -- Second content chunk is a Byte
\5  -- Third content chunk is a String
\15 -- Fourth content chunk is an Array
\0  -- Division character because the number of arguments is unknown
\0xFE \0xD3 -- Int16 => 65235
\25 -- Byte => 25
\11 -- Total length of string chunk is 11 bytes
\0\72 -- String[0] => 'H' (\0 needed because of unicode)
\0\101 -- String[1] => 'e'
\0\108 -- String[2] => 'l'
\0\108 -- String[3] => 'l'
\0\108 -- String[4] => 'o'
\10 -- Total length of array is 10 bytes
\0 -- Array length / array argument specifier division
\2 -- First array content chunk is a Byte
\14 -- Second array content chunk is a Ticket
\0 -- Array argument specifier / array content division
\42 -- First array content chunk (Byte) is the answer to all questions
\0x12\0x34\0x56\0x78 -- Ticket => 305419896

A note for length: only seven of eight bits may be used in length specifiers,
the least significant bit is always 1. Without this rule a length could contain
a zero byte which would cause problems with the division characters.

Ticket meta information is not send along, it has to be requested (this is
efficient since the same ticket is used in many RPC strings). When a ticket goes
out of order this must be told to any party that knows that ticket.

If you can't see the method name you're right, in this example there is no
method name. In a normal situation you would send over an RPC string with the
information of who you want to reach in the arguments and the argument list as
an array thus something like: |who-to-call|arguments|.

The details of the system have to be worked out, but I'm confident that this
will make inter language calls much easier.

Note that if basic data can't be translated without lossage into a basic data
type in this system (an example is a WebScheme Rational that can't be translated
into a double without losing exactness) the sending language has to decide what
to do (for Rational that means losing exactness).

Greetings,

Peter