Re: Channel sessions

[olafBuddenhagen]

Hi,

On Mon, Aug 13, 2007 at 03:08:27PM +0200, Carl Fredrik Hammar wrote:
> <olafBuddenhagen@gmx.net> writes:

> I will use channel for the object that a client interacts with, i.e.
> one is created whenever the translator is opened.

That is what I call a channel instance. You can also call it a channel
session; but referring to an individual instance only as "channel" would
be very confusing IMHO.

> > A channel module is a single unit implementing the channel
> > interface, running within libchannel or in a channel translator. It
> > has a client side, which can be access through a filesystem node or
> > through libchannel; and a device side, which can access either a
> > Mach device, or the client side of another channel module. A channel
> > stack is a layering of several channel translators/modules.
> 
> This object I have opted to call a hub, ``module'' would perhaps be a
> bit better.  However, I don't want to stray to much from libstore's
> terminology.  And in libstore module is the shared object containing a
> class that can be loaded dynamically.

Hm... Haven't distinguished between these up till now. Is it necessary
to avoid ambiguity?...

> A type is simply the identifier of a class.  A class is the entity
> implementing a type, similar to its use in OOP languages, with methods
> and other constant data.  Type and class is sometimes used
> interchangeably.

Well, I can't make much of OO terminology; I'm still not really sure
what you actually mean by a channel class... I hope you don't mind if I
don't use these terms :-)

> > Why can't --buffer or --no-buffer be used? In storeio, you also have
> > a number of optional flags, and if you give several of them, you get
> > implicit layering, although you only set a single translator --
> > without using the "unwieldy" layering syntax...
> 
> Yes, that is an option in many cases.  But for buffering, position in
> the stack may be crucial.  Consider a tee, buffer, device stack, where
> the tee will only open a single channel to the buffer.

Well, when implicitely selecting several modules where order is relevant
(which I think can happen with storeio as well), simply establish a
fixed order; if for some reason a different order is necessary, one can
still stack several translators, or use the layering syntax...

BTW, I think junctions probably should never be specified with some
options, but rather always by explicit translator stacking.

> If the buffer were on top, there would have to be one buffer per
> channel, which is redundant since the same stream is passing through
> all of them.

I think there are cases where having individual buffers per instance
could be desirable.

> > Also, I thing there still might be some confusion here. It seems
> > very important that we get a common understanding of the topology.
> >
> > There are basically two kinds of junctions regarding the topology:
> > Multi-client junctions and multi-device junctions. Those are two
> > totally different things, serving very different purposes, and
> > usually not used together.
> 
> In not sure if this is what you were getting at, but your talk of
> topology got me thinking.  When faced with a back-end junction I was
> assuming that one only wants one channel per back-end, leaving only
> one channel to be shared by clients.  This forced me to choose
> front-end junction as well, an arbitrary choice, with in-tee being the
> closest at hand.
> 
> However, it is now clear to me that when a back-end junction channel
> is opened, new channels to all back-ends should be opened.  By
> layering a front-end junction over the back-end junction one can still
> have only one channel per back-end.

Yes, that's exactly what I was getting at. Good to have this sorted out
at last :-)

> > For each of these base kinds, there are two diretions (input our
> > output); for bi-directional channels, usually a pair is necessary.
> > However, the pairing is not always obvious. In your example, we had
> > two clients that read from the same audio device. What we need here
> > in the tee I desribed, which is a muliti-client input junction. What
> > is the complementary multi-client output junction? In the audio
> > case, probably an arithmetic mixing makes most sense -- the tee
> > allows multiple clients to simultaneously read from the same device,
> > and the mixer allows multiple clients to simultaneously output to
> > the same device. In other cases however, like networking for
> > example, we would obviouly need something else to provide
> > complementary functionality...
> 
> A mixer would be the most appropriate way to funnel output streams
> into a single one.  But that obviously only works for audio.

Exactly.

> For a general purpose front-end junction, the simplest and most
> neutral way would be used, which would be a fifo (or perhaps it should
> be called a fofi. ;-))

I think there might be some cases, where some variant of fifo is most
appropriate; but I'm not at all convinced that it's the most useful in
the general case. In fact, my point was that there might not be such a
thing as a general case here... Useful behaviour is just too
type-specific I think.

> > Now what about the out-tee? This is somewhat confusing: While the
> > in-tee is a multi-client input junction, the out-tee is a
> > multi-device output junction -- something totally unrelated with
> > very different use cases. Also, althouh the basic idea of
> > duplicating streams is the same, forwarding from one client to
> > multiple devices is totally different implementation-wise than
> > forwarding from one device to multiple clients...
> 
> Yes, I am well aware of the differences.  The out-tee is much, much
> simpler.  Handling the case when some channels of an in-tee aren't so
> that other channels don't stall is tricky.

I'm not sure there is any difference in this regard. I don't see why
backends should provide any more guarantees about not stalling than
frontends...

> > The situation is even more confusing for fifo. There are two
> > distinct kinds: A split-fifo (with a topology like the tee), and a
> > merge-fifo; each having in- and out-variants. (Maybe better leave
> > out the "fifo" alltogether and give them totally distinct names
> > instead, to reduce confusion...)
> 
> What would a merge-fifo be?  I don't think we've brought that one up
> before.  I have always been referring to split-fifo (I think.)

Well, if we want to do network load balancing over multiple devices for
example, we'd need some variant of a multi-backend out-fifo, which is a
splitting fifo, as it takes packets from one stream and distributes them
to several. When receiving packets from multiple devices, we need a
multi-backend in-fifo, which is a merging fifo, as it interleaves
packets from several streams into one.

> One more thing to think about is making sure that when a in and out
> channel is rejoined that they belong to the same client.  This might
> very well happen in a naive implementation of a joiner, that simply
> pairs the last in channel with the next out channel or vice versa.
> Considering that it is possible to open read or write only channels.
> 
> In fact I think this rules out option one and three.  A modified
> version of three might work though, where the split and join is in a
> single module that maintains two separate stacks.  Lets call this
> option four.
> 
> Another way to tackle this would be to supply a client id along with
> the channel when it is opened, allowing the relevant party to pair up
> channels.  This would allow option one and three to be implemented.
> But the client id can't be forwarded across translators (unless we
> expand the IPC protocol,) so channel would have to be paired up before
> this point.

Indeed, I haven't considered this issue at all so far. I'll have to
think more about it I guess. Client IDs seem like a reasonable approach
at first glance at least...

> libchannel also handles control interfaces, analogous to ioctls.  This
> just means that channels can handle any IPC messages not recognized by
> the translator.  It just occurred to me that one might want to split
> them into separate paths as well.  But this can be handled similarly
> to in/out splitting.

Right. I'm not sure how useful this might be in practice, but the
generalization should certainly help thinking about the pairing issue.

-antrik-