I've looked through the code more and more, along with twisted. From what
I can see, these are the primary advantages from a code refactoring into the
twisted framework:
1. The HTTP interface is much nicer to work with...the conversion to a web
interface will be somewhat easier.
2. The existing structure is heavily threaded. This would go away and be
replaced with a single-threaded reactor-style system. Deferred's will replace
a lot of the thread-glue logic. And no more locks (not necessary since there
are no thread conflicts).
3. Adding features that are already coded in twisted such as e-mail or
instant-messaging type interfaces becomes trivial.
4. The application interface becomes available which automatically handles
hot-replugging of the code (basically, edit the code as it is running instead
of rebooting the daemon), and handles persistent option storage trivially.
5. A pluggable interface where the various features such as the download
manager, chat interface, gossip interface, etc., can be loaded/unloaded at
will. And adding additional features becomes easy.
The only real hassle that I can see is unravelling the threading that is
written into basically everything.
One question that I see. Why oh why would you want to switch from a Chord
system to Kademlia?
Kademlia's primary advantage is that it is less structured than Chord. The
proponents claim that it is more robust to node failures. However, from what
I've seen, the major problem is that Kademlia's lack of structure makes it
extremely prone to divide-and-conquer or surround-and-conquer type attacks.
It also seems to perform just as well as Chord so other than the fact that
the "maintenance protocol" is basically piggybacked onto searches, it seems
to have no inherent advantages that I can see.
Personally, I'm leaning towards a Symphony-style structure. This converts
the network from a O(log N) diameter with log N links to one with either the
same diameter and constant (small) number of links, or to O(log N/log log N)
diameter with log N links.
Symphony is very simple. With Chord, you have to keep track of neighbors at
specific non-random points on the circle. With Symphony, the neighbors are
random and just chosen from a specific distribution. Or with one of the slight
modifications, simply chop the ring up into K distinct pieces (with K
neighbors) and randomly pick any node from each piece. Nodes pass their lists
of neighbors around, and routing is done while looking at not only the
links but also the neighbors of the links. This second pass takes very little
extra communication overhead but allows the above mentioned O(log N/log log N)
diameter (which is the minimum possible diameter with log N links). Since
regular Chord has a fixed structure, the neighbors-of-links trick doesn't
work (no short paths). Also, individual users can trivially tune the
performance (within limits) of their own nodes by simply changing the K
parameter. Higher numbers of links results in shorter paths, while lowering
the number of links reduces communication overhead.
Since specific neighbors don't matter, a node can also optimize link
round trip times. Simply contact a potential neighbor (as before) and ask for
the neighbors of that neighbor. Check response times to each of those and
choose the shortest one as a link.
By the way, there's also a randomized Chord variant. Instead of picking the
node at N/2, use N/2-N, N/4-N/2, N/8-N/4, etc. Any node in that range works.
And the advantage is that the above-mentioned neighbors-of-links trick works,
and you can also do round-trip-time optimization. The disadvantage is that
it's not quite as simple as Symphony and that the tunable neighbor trick is
lost.
If you want the Kademlia structure still, there's another one out of France
that simplifies Kademlia and picks up some of the advantages of the Symphony
structure. It's called "Broose". It essentially models the De Brujin network
(the theoretical lowest diameter network) in a Kademlia framework, which
picks up the O(log N/log log N) diameter advantage. Also Broose only contains
two buckets of nodes to track vs. several more in Kademlia.