[Mldonkey-users] A new feature

[Thoran]

Hi list,
There is something I would like to implement. But because it will take
some time to code, I would like to know if it has a chance to be
accepted.  

The idea came to me when I was downloading a file on edonkey that had
"mutated".  I'm sure you guess what it means:  when I sent my request,
I received several answers for files with exactly the same size but not
the same MD4.  When files are more than 500 Mo, I think we are pretty
sure that they were once the same file, before a "mutation" occured.

I started downloading both of them (the file, and the mutated file
(let's call it, ...mmm,  the X-file ;-)).
I observed, by looking at the list of md4's chunks, that all the
chunks were the same except one.

The X-file was barely complete (just 20 %).  The original file was
complete. 

Now a strange phenomenon occured.  Exactly the same chunks we
downloaded on both files.  It looked like as soon as a chunk was
downloaded into one of the two files, it got automatically downloaded
into the other.  The lists of downloaded chunks were constantly the
same for both files (from the gui report).

At a moment, the  chunks that had mutated was downloaded.  And, as
expected, it didn't appeared into the X-file.  But the X-file, when I
canceled it, reached more than 80% of completeness (far more than the
initial 20%)

Could a developper explain me if this behaviour was expected?  And
what exactly happened.

For what I can understand, there are two possibilities (I don't know
the edonkey protocol in great details yet, otherwise, I could decide
which one is correct)

1)
The client ask for a given chunk by its md4 (the md4 of the chunk),
then all the possessors of this chunk can be uploaders, even if they
are not uploading the same file.  At this moment, there is some kind
of autoconnection of mlnet to itself such that when the chunk is
downloaded into a file, it gets instantly downloaded into the X-file
(But apparently, there are some verifications to avoid
autoconnection).

2)
mlnet discovers itself that a chunk *is* in several files and then
it dupplicates it (But I can't find the code responsible for that).

3)
mlnet get confused by the fact that the same chunk can be in
different files.  Thus, what the gui reports is bullshit.

Anyway, this is a good idea.  I would like to add an option
(smart_download?) that would download all file by trying to get chunks
from all its mutants.  This could be very efficient, especially for
rare files.

The file I was downloading was in this case.  Between the file and the
X-file, the later had more sources.  This does not surprise
me.  The poor loosers that are downloading this very incomplete file
had all reached the 20% of completeness and are now waiting forever
for the remaining.  They are motivated.
On the contrary, when the winners finish downloading the original (and
complete) file, they erased it from their shared list.

Finally, there are more loosers than winners.  I propose to crush the
loosers even further by using them as sources for chunks they have and
didn't mutate.

This is the first part of my project and it should be quite easy to
code (simply because it already seems to work!!!).

The second part is to extend this idea to Fasttrack.  A lot of files
are common between edonkey and fasttrack.  Suppose we are downloading
on both network the same file.  We take care the two plugins  download  disjoint
parts of the file.  When Fasttrack seems to have complete a donkey
chunk, we compute its donkey md4, and if it matched what donkey is
downloading we directly add it to the donkey download.  And by
symetry, we do the same in the other direction.

This will be far more technical to code because those two plugins are
not design to communicate between each other at this very low level.

What do you think of this ?

  
Cheers,

 Thoran