Re: [circle] How are data items stored/forwarded?

[Paul Campbell]

On Mon, Oct 04, 2004 at 12:27:05PM -0400, rodrigo benenson wrote:
> now that you are involved in the Circle twistified rewrite maybe 
> rereading 
> http://lists.gnu.org/archive/html/circle-discuss/2004-05/msg00028.html
> 
> could give you some ideas.

Been thinking about it. See, here's the issue:

The core is fine. Everyone can agree on what that represents. In fact, at a
lower level, I've been thinking that even mixed TCP/UDP packet formats can
work. My current RPC interface is actually pretty darned trivial from the
Node class point of view. It implements just one call ("call"). Call takes
the existing thecircle node.call() semantics except that it always returns
a deferred (since the threaded interface is going away). For handling incoming
requests, it converts everything back into a high level structure
(deserializing) and then makes a single call back into the DHT code:
self.node.perform_request(payload, address, nonce)

where payload is the query structure, address is the usual (address, port)
pair, and nonce is the unique nonce passed from the caller. The nonce isn't
really necessary to expose. But I figured that it might be necessary to be
stateful some time in the future (I hope not!). It treats the call as a
deferred and expects a single return value (the result, which can be any
safe_pickle'able structure), including an error structure.

To move this into a TCP/UDP format, I can create something like this:

class IMessageTransport(components.Interface):
    """The message RPC transport interface."""
    def sendMessage(self, address, nonce, reqflag, data, timeout, retries):
        """Sends a message with the given nonce.
           reqflag indicates if it is an RPC request(0) or response(0).
           data is a string (the serialized data to send).
           timeout is the number of seconds to wait for a response.
           retries is the number of times to retry the same request.
           address is who to send it to (IP, port).
        """
    def recvRequest(self, address, nonce, data):
        """A call defined by the subclassing code. It gets called whenever
            a remote node calls this one. The node should process the
            request in the data field, and then call sendMessage with the
            appropriate response.
        """

Then we can declare any number of implementations of the interface. Three that
I can think of right off are a UDP interface, a TCP interface, and of course
a proxy interface. In fact, a fourth could be a TCP NAT-busting proxy
interface that does the "3-way" trick (pound the NAT with TCP SYN's both
externally and internally so that it opens the port up). Whether these are
implemented as components or adapters is sort of personal choice.

However, once you move away from the core, the question is how to represent
the higher level components. By that I mean what specifically should a
"service" define? It seems to me that it will end up with some rather
weak (and loose) definitions for an IDHTService such as:

class IDHTService(components.Interface):
    """A service to the DHT system."""
    def start(self):
        """Handles startup functions for the service (registering itself).
           Mostly, this should be registering handlers for both RPC processing
           and handlers for GUI interfacing."""
    def stop(self):
        """Handles shutting down a service."""
    def go_online(self):
        """Handles transition from offline to online."""
    def go_offline(self):
        """Handles transition from online to offline."""

I changed the "IService" word to "IDHTService" since the name IService is
already taken (as part of the twisted application functions).

Then you'd have something like the following:

class IGTKService(components.Interface):
    """Hooks for the GTK implementation. Adapters convert an IDHTService
       into an IGTKService."""
class IHTMLService(components.Interface):
class ITextService(components.Interface):
class IServerService(components.Interface):
    """Hooks to PB to present a remote server service, which then gets
        adapted to an IDHTService, which can then adapt to the GUI's."""
...