2010/3/27 Carlo von Loesch
<address@hidden>
Blaine Cook typeth:
| When doing distribution of messages to 10,000 or 100,000 recipients,
| you want to be able to batch those deliveries (PSYC's notion of
| multicast) and more importantly you want low-latency, high reliability
| response characteristics. If each delivery of 10,000 takes even 1ms,
| the total delivery time is going to be 10 *seconds*. A more realistic
Uhm no, PSYC's notion of multicast would be, you send the notice out
to a dozen already established TCP links which redistribute it to
another dozen each. Something like that. And you try to do it close
to simultaneously using non-blocking writes and connects where
necessary. A tree of 12/12 is just an example.
This really sounds very neat.
I think the great advantage of the AGPL is that anyone can take the base code (im assuming the first cut will have elements of PHP/HTTP, capable of running on a low end server) and if they dont like something about it, they can make a 'super-charged' version using for example, other transport layers (psyc/udp/multicast/websocket -- yes i think php does udp), other programming languages e.g. scala 2.8, different API support, features like DDNS/bind9 proxying tahoe-lafs etc. and contribute the changes back to the community.
But you need to start somewhere, so I think Matt is making a good call to start with a LAMP style stack, similar to GNU FM, that you can really push to the limits, before inventing something new. A clean architecture on the programming side (perhaps we're looking at something like symfony2 -- tho you need to be careful in design of MVC make the M portable and the C obeying standards, the V follows), on the data side (using linked data principles) and on the license side (AGPL), should allow great flexibility.
| *MOST* HTTP servers are terribly ill-suited to this sort of messaging
| --- the sort that social networks promote. HTTP response times are
| normally more like 30ms if you're lucky, 1-2s if you're not. So in
| that regards, I agree, HTTP is ill-suited to this sort of work.
Alright. So we have overhead, a TCP connection for each direction, and
the problem of dealing with existing HTTP implementations. Which HTTP
implementations are generally in place in commodity web hosting sites?
| on. HTTPs stateless nature means that massive parallelism works out of
| the box - not something that can be said for e.g., XMPP or IRC. Most
| XMPP servers will fail to successfully negotiate an s2s link unless
| both servers are clustered at the back-end (thanks to dial-back), an
| approach which is far from scalable in most XMPP servers. I love XMPP,
| but don't ignore the fact that it's not a panacea, either ? no
| protocol is.
I'm notable for my criticism of XMPP scalability. Almost needless to
say, you don't have this problem with PSYC. The problem you do still
have with HTTP and PSYC is that parallelism still has to go somewhere
to pick the data sets for each particular person, but the problem and
the solution would be the same no matter which protocol is picked.
PSYC is more likely to be able to do without a "middleware" kind of
approach, by having people addressed at the right server right away -
but that's just another option. And you could do that with HTTP too,
if you really wanted to.